DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on August 20, 2026 has been entered.
Response to Amendment
Claims 1, 6, 9 and 14 have been amended changing the scope and contents of the claim.
Claims 21-24 have been newly added.
Claims 17-20 have been cancelled.
Applicant’s amendment filed August 20, 2026 overcomes the following objection/rejection(s) from the last Office Action of March 20, 2026:
Interpretation of the claims under 35 U.S.C. 112(f)
Rejections to the claims under 35 USC § 102
Response to Arguments
Claims 1 and 9:
Applicant’s arguments with respect to claim(s) 1 and 9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claims 6 and 14:
Applicant's arguments field August 20, 2026 have been fully considered but they are not persuasive.
Speciifcally, Applicant argues, “With respect to claims 6 and 14 in particular, these claims have been amended to recite that an opacity of the one or more color codes "is based on a determined severity of the at least one of the pathology and the musculoskeletal area." Carter, relied upon for these claims, assigns a color and/or opacity to a bounding region based on a machine-learning confidence score (Carter, col. 10, 1. 48; col. 12, 1. 53), not based on a determined severity of the pathology or musculoskeletal area. (Remarks, 9).”
The examiner respectfully disagrees. As noted in Carter and below at column 10, line 48, “At block 406, for the given image region currently being processed, the medical provider system 102 may determine one or more bounding shape display parameters (such as color, opacity and/or shape type) based at least in part on a label within the metadata for the given region. The label may represent or specify a specific pathology or other classification previously determined by a machine learning model and assigned as a classification label to the given region. In some embodiments, for instance, different pathologies may be assigned different bounding shapes, colors or other display parameters, which may be configurable by a user. In one example, at least one display parameter determined at block 406 may be based on a confidence level determined by one or more models. For example, a specific color and/or opacity may be assigned to the bounding region based on its confidence score, as will be further discussed below.” The examiner takes the position that different pathologies can be read under the broadest reasonable interpretation as different stages/severities of disease. For example, a cancer at stage I and stage II are read to be different pathologies, of which can be coded in different colors.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7, 9 15, 21-24 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2020/0402237 to Song et al. (hereinafter Song), and further in view of De-la-Cruz-Torres, B.; Romero-Morales, C. Muscular Echovariation as a New Biomarker for the Classification of Soleus Muscle Pathology: A Cross-Sectional Study. Diagnostics 2021, 11, 1884. https://doi.org/10.3390/diagnostics11101884 (hereinafter Torres).
Regarding independent claim 1, Song discloses A method (abstract, “Embodiments of the disclosure provide systems and methods for generating a diagnosis report based on a medical image of a patient.”), comprising:
receiving a plurality of images from an ultrasound probe (paragraph 0015, “Consistent with the present disclosure, diagnosis report generating system 100 may receive medical images 102 from image acquisition device 101;” paragraph 0016, “In some embodiments, image acquisition device 101 may acquire medical images 102 using any suitable imaging modalities, including, e.g., functional MRI (e.g., fMRI, DCE-MRI and diffusion MRI), Cone Beam CT (CBCT), Spiral CT, Positron Emission Tomography (PET), Single-Photon Emission Computed Tomography (SPECT), X-ray, optical tomography, fluorescence imaging, ultrasound imaging, and radiotherapy portal imaging, etc.”);
analyzing, using an artificial intelligence system, the plurality of images to determine one or more characteristics in each image of the plurality of images (paragraph 0026, “CAD unit 122 may further segment the images to identify different regions of interest (e.g., anatomical structures) in the image, e.g. heart, lung, ribcage, blood vessels, possible round lesions. Various segmentation methods may be used, including, e.g., matching with an anatomic databank, or using neural networks trained using sample images. The identified structures may be analyzed individually for special characteristics.”);
identifying, by the artificial intelligence system, at least one of a pathology shown in the plurality of images and a musculoskeletal area shown in the plurality of images (paragraph 0027, “If the detected structures meet certain criteria, CAD unit 122 may highlight them in the image for the radiologist, for example, using boundary contours or bounding boxes. This allows the radiologist to draw conclusions about the condition of the pathology. In some embodiments, CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or he contrast characteristics of a tumor.”);
automatically generating a report that includes the at least the one of the pathology and the musculoskeletal area (Figure 4, element S412, “automatically construct a diagnosis report;” paragraph 0052, “ For example, the report may include a patient information section 212/312 showing patient name, gender, and age, as well as examination section 214 containing scan information derived from the patient's meta data. Report generation unit 124 may further generate diagnosis content of the report based on step S408. For example, the diagnosis report may include impression section 216/316 and findings section 218/318. The diagnosis sections in the report may include screenshots of images imported from the CAD analysis as well as text information indicating, e.g., the type of the detected object (i.e. bleeding type cerebral hemorrhage), the position of the detected object (i.e. left frontal lobe), and parameters calculated in step S410. ”); and
providing the report to a computing device (abstract, “The system also includes a display configured to display the diagnosis report;” paragraph 0035, “Processor 120 may render visualizations of user interfaces to display data on a display 130. Display 130 may include a Liquid Crystal Display (LCD), a Light Emitting Diode Display (LED), a plasma display, or any other type of display, and provide a Graphical User Interface (GUI) presented on the display for user input and data display. ”).
Song fails to explicitly disclose as further recited. However, Torres discloses wherein the at least one of the pathology and the musculoskeletal area is identified by comparing a determined echotexture of a musculoskeletal area shown in the plurality of images to an echotexture of comparable musculoskeletal areas in a plurality of stored images (abstract, “ Echotexture characteristics of soleus muscle were reviewed for 84 subjects. They were divided based on the muscle echogenicity in three groups (Injury Type 1 group, Injury type 2 group and healthy group). Echointensity (EI) and Echovariation (EV) were taken in all groups like quantitative US variable. Results. The Injury Type 1 group was identified by a hypoechoic area and characterized by a higher EV; and Injury Type 2 group was identified by a fibrotic area and characterized by a lower EV. The echogenic pattern of healthy people obtained an intermediate value of EV between both injured soleus types. Conclusions. EV may be useful to classify different types of soleus muscle pathology according to the echogenicity pattern.” See also Figure 2 where the region is detected as the yellow mark in the center images, the region around that is considered muscle; the differentiation between healthy and injured is performed based on the echotexture features)
Song is directed toward, “ provide systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Torres is directed toward “to analyze the most frequent echotexture findings of patients with soleus muscle injury, located in the central intramuscular tendon (IMT), and healthy people to determine whether they behave differently and to propose an ultrasound (US)-based classification (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand diagnosis and medical reporting are often done based on baseline data; said differently, detecting abnormal patterns is often performed based on comparing a patient to a normal data set to detect differences. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to ensure deviations of a patient in question from a healthy patient can be detected for diagnosis.
Regarding dependent claim 7, the rejection of claim 1 is incorporated herein. Additionally, Song discloses wherein the one or more characteristics include one or more patterns identified within one or more images of the plurality of images (paragraph 0027, “CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or the contrast characteristics of a tumor.”), one or more shapes identified within the one or more images of the plurality of images (paragraph 0049, “ In some embodiments, CAD unit 122 may be used to perform a CAD analysis to detect the conspicuous object;” detecting the object is read as detecting the shape); an echogenicity identified in one or more images of the plurality of images (paragraph 0027, “CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or he contrast characteristics of a tumor;” pixel intensity and contrast characteristics correlate to brightness which is an indication of echogenicity).
Torres discloses identifying an echotexture in one or more images of the plurality of images (abstract, “Echotexture characteristics of soleus muscle were reviewed for 84 subjects.”).
One of ordinary skill in the art before the effective filing date of the claimed invention would be aware there are numerous alternative features that can be detected from images to quantify different disease states. Further, echotexture is well known by one of ordinary skill in the art before the effective filing date of the claimed invention to be a value characterizing the appearance of tissues/organ internal structures as being homogenous or heterogenous. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to quantify additional features of the ultrasound images to quantify a wider variety of disease states.
Regarding independent claim 9, the rejection of claim 1 applies directly. Additionally, Song further discloses A system (abstract, “Embodiments of the disclosure provide systems and methods for generating a diagnosis report based on a medical image of a patient. ”), comprising:
at least one processing unit (Figure 1, element 120, “processor”); and
a memory operably coupled to the at least one processing unit (Figure 1, element 150, “memory”) and storing instructions that, when executed by the at least one processing unit, perform operations (paragraph 0008, “Embodiments of the disclosure further provide a non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, causes the one or more processors to perform a method for generating a diagnosis report based on a medical image of a patient.”), comprising:
receiving a plurality of images captured by an ultrasound probe (paragraph 0015, “Consistent with the present disclosure, diagnosis report generating system 100 may receive medical images 102 from image acquisition device 101;” paragraph 0016, “In some embodiments, image acquisition device 101 may acquire medical images 102 using any suitable imaging modalities, including, e.g., functional MRI (e.g., fMRI, DCE-MRI and diffusion MRI), Cone Beam CT (CBCT), Spiral CT, Positron Emission Tomography (PET), Single-Photon Emission Computed Tomography (SPECT), X-ray, optical tomography, fluorescence imaging, ultrasound imaging, and radiotherapy portal imaging, etc.”);
providing the plurality of images to an artificial intelligence system (paragraph 0026, “CAD unit 122 may further segment the images to identify different regions of interest (e.g., anatomical structures) in the image, e.g. heart, lung, ribcage, blood vessels, possible round lesions. Various segmentation methods may be used, including, e.g., matching with an anatomic databank, or using neural networks trained using sample images. The identified structures may be analyzed individually for special characteristics.”);
causing the artificial intelligence system to analyze the plurality of images to identify at least one of a pathology shown in the plurality of images and a musculoskeletal area shown in the plurality of images (paragraph 0027, “If the detected structures meet certain criteria, CAD unit 122 may highlight them in the image for the radiologist, for example, using boundary contours or bounding boxes. This allows the radiologist to draw conclusions about the condition of e pathology. In some embodiments, CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or he contrast characteristics of a tumor.”);
generate a report that includes the at least the one of the pathology and the musculoskeletal area (Figure 4, element S412, “automatically construct a diagnosis report;” paragraph 0052, “ For example, the report may include a patient information section 212/312 showing patient name, gender, and age, as well as examination section 214 containing scan information derived from the patient's meta data. Report generation unit 124 may further generate diagnosis content of the report based on step S408. For example, the diagnosis report may include impression section 216/316 and findings section 218/318. The diagnosis sections in the report may include screenshots of images imported from the CAD analysis as well as text information indicating, e.g., the type of the detected object (i.e. bleeding type cerebral hemorrhage), the position of the detected object (i.e. left frontal lobe), and parameters calculated in step S410. ”); and
provide the report to a computing device (abstract, “The system also includes a display configured to display the diagnosis report;” paragraph 0035, “Processor 120 may render visualizations of user interfaces to display data on a display 130. Display 130 may include a Liquid Crystal Display (LCD), a Light Emitting Diode Display (LED), a plasma display, or any other type of display, and provide a Graphical User Interface (GUI) presented on the display for user input and data display. ”).
Song fails to explicitly disclose as further recited. However, Torres discloses wherein the at least one of the pathology and the musculoskeletal area is identified by comparing a determined echotexture of a musculoskeletal area shown in the plurality of images to an echotexture of comparable musculoskeletal areas in a plurality of stored images (abstract, “ Echotexture characteristics of soleus muscle were reviewed for 84 subjects. They were divided based on the muscle echogenicity in three groups (Injury Type 1 group, Injury type 2 group and healthy group). Echointensity (EI) and Echovariation (EV) were taken in all groups like quantitative US variable. Results. The Injury Type 1 group was identified by a hypoechoic area and characterized by a higher EV; and Injury Type 2 group was identified by a fibrotic area and characterized by a lower EV. The echogenic pattern of healthy people obtained an intermediate value of EV between both injured soleus types. Conclusions. EV may be useful to classify different types of soleus muscle pathology according to the echogenicity pattern.” See also Figure 2 where the region is detected as the yellow mark in the center images, the region around that is considered muscle; the differentiation between healthy and injured is performed based on the echotexture features)
Song is directed toward, “ provide systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Torres is directed toward “to analyze the most frequent echotexture findings of patients with soleus muscle injury, located in the central intramuscular tendon (IMT), and healthy people to determine whether they behave differently and to propose an ultrasound (US)-based classification (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand diagnosis and medical reporting are often done based on baseline data; said differently, detecting abnormal patterns is often performed based on comparing a patient to a normal data set to detect differences. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to ensure deviations of a patient in question from a healthy patient can be detected for diagnosis.
Regarding dependent claim 15, the rejection of claim 9 is incorporated herein. Additionally, Song discloses wherein analyzing the one or more images includes identifying one or more patterns within one or more images of the plurality of images (paragraph 0027, “CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or the contrast characteristics of a tumor.”), identifying one or more shapes within the one or more images of the plurality of images (paragraph 0049, “ In some embodiments, CAD unit 122 may be used to perform a CAD analysis to detect the conspicuous object;” detecting the object is read as detecting the shape); identifying an echogenicity in one or more images of the plurality of images (paragraph 0027, “CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or he contrast characteristics of a tumor;” pixel intensity and contrast characteristics correlate to brightness which is an indication of echogenicity)
Torres discloses identifying an echotexture in one or more images of the plurality of images (abstract, “Echotexture characteristics of soleus muscle were reviewed for 84 subjects.”).
One of ordinary skill in the art before the effective filing date of the claimed invention would be aware there are numerous alternative features that can be detected from images to quantify different disease states. Further, echotexture is well known by one of ordinary skill in the art before the effective filing date of the claimed invention to be a value characterizing the appearance of tissues/organ internal structures as being homogenous or heterogenous. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to quantify additional features of the ultrasound images to quantify a wider variety of disease states.
Regarding dependent claim 21, the rejection of claim 1 is incorporated herein. Additionally, Torres discloses further comprising color coding, based on the identified musculoskeletal area, a portion of one or more images of the plurality of images that corresponds to the musculoskeletal area, wherein different musculoskeletal areas are associated with different colors (Figure 2, “Middle panel: ultrasound image with marked lesion area (yellow line)” the unhealthy region of the musculoskeletal area is associated with the yellow marking ).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand color coding an image for review by a user can aid in an understanding of the image itself; said differently, color coding can provide information as related to size, location, etc of a region upon review without a user having to perform that analysis themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to increase reviewer efficiency and understanding.
Regarding dependent claim 22, the rejection of claim 9 is incorporated herein. Additionally, Torres discloses wherein the operations further comprise color coding, based on the identified musculoskeletal area, a portion of one or more images of the plurality of images that corresponds to the musculoskeletal area, wherein different musculoskeletal areas are associated with different colors (Figure 2, “Middle panel: ultrasound image with marked lesion area (yellow line)” the unhealthy region of the musculoskeletal area is associated with the yellow marking ).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand color coding an image for review by a user can aid in an understanding of the image itself; said differently, color coding can provide information as related to size, location, etc of a region upon review without a user having to perform that analysis themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to increase reviewer efficiency and understanding.
Regarding dependent claim 23, the rejection of claim 1 applies directly. Additionally, Song further discloses A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations (paragraph 0008, “Embodiments of the disclosure further provide a non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, causes the one or more processors to perform a method for generating a diagnosis report based on a medical image of a patient. ”) comprising:
receiving a plurality of images from an ultrasound probe (paragraph 0015, “Consistent with the present disclosure, diagnosis report generating system 100 may receive medical images 102 from image acquisition device 101;” paragraph 0016, “In some embodiments, image acquisition device 101 may acquire medical images 102 using any suitable imaging modalities, including, e.g., functional MRI (e.g., fMRI, DCE-MRI and diffusion MRI), Cone Beam CT (CBCT), Spiral CT, Positron Emission Tomography (PET), Single-Photon Emission Computed Tomography (SPECT), X-ray, optical tomography, fluorescence imaging, ultrasound imaging, and radiotherapy portal imaging, etc.”);
analyzing, using an artificial intelligence system, the plurality of images to determine one or more characteristics in each image of the plurality of images (paragraph 0026, “CAD unit 122 may further segment the images to identify different regions of interest (e.g., anatomical structures) in the image, e.g. heart, lung, ribcage, blood vessels, possible round lesions. Various segmentation methods may be used, including, e.g., matching with an anatomic databank, or using neural networks trained using sample images. The identified structures may be analyzed individually for special characteristics.” paragraph 0027, “If the detected structures meet certain criteria, CAD unit 122 may highlight them in the image for the radiologist, for example, using boundary contours or bounding boxes. This allows the radiologist to draw conclusions about the condition of e pathology. In some embodiments, CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or he contrast characteristics of a tumor.”);
identifying, by the artificial intelligence system, at least one of a pathology shown in the plurality of images and a musculoskeletal area shown in the plurality of images (paragraph 0027, “If the detected structures meet certain criteria, CAD unit 122 may highlight them in the image for the radiologist, for example, using boundary contours or bounding boxes. This allows the radiologist to draw conclusions about the condition of e pathology. In some embodiments, CAD unit 122 may further determine one or more parameters that quantify the medical condition. For example, the parameters may include size (such as diameter, length, width, depth, etc.), volume, pixel intensities, or he contrast characteristics of a tumor.”),
automatically generating a report that includes the at least the one of the pathology and the musculoskeletal area (Figure 4, element S412, “automatically construct a diagnosis report;” paragraph 0052, “ For example, the report may include a patient information section 212/312 showing patient name, gender, and age, as well as examination section 214 containing scan information derived from the patient's meta data. Report generation unit 124 may further generate diagnosis content of the report based on step S408. For example, the diagnosis report may include impression section 216/316 and findings section 218/318. The diagnosis sections in the report may include screenshots of images imported from the CAD analysis as well as text information indicating, e.g., the type of the detected object (i.e. bleeding type cerebral hemorrhage), the position of the detected object (i.e. left frontal lobe), and parameters calculated in step S410. ”); and
providing the report to a computing device (abstract, “The system also includes a display configured to display the diagnosis report;” paragraph 0035, “Processor 120 may render visualizations of user interfaces to display data on a display 130. Display 130 may include a Liquid Crystal Display (LCD), a Light Emitting Diode Display (LED), a plasma display, or any other type of display, and provide a Graphical User Interface (GUI) presented on the display for user input and data display. ”).
Song fails to explicitly disclose as further recited. However, Torres discloses wherein the at least one of the pathology and the musculoskeletal area is identified by comparing a determined echotexture of a musculoskeletal area shown in the plurality of images to an echotexture of comparable musculoskeletal areas in a plurality of stored images (abstract, “ Echotexture characteristics of soleus muscle were reviewed for 84 subjects. They were divided based on the muscle echogenicity in three groups (Injury Type 1 group, Injury type 2 group and healthy group). Echointensity (EI) and Echovariation (EV) were taken in all groups like quantitative US variable. Results. The Injury Type 1 group was identified by a hypoechoic area and characterized by a higher EV; and Injury Type 2 group was identified by a fibrotic area and characterized by a lower EV. The echogenic pattern of healthy people obtained an intermediate value of EV between both injured soleus types. Conclusions. EV may be useful to classify different types of soleus muscle pathology according to the echogenicity pattern.” See also Figure 2 where the region is detected as the yellow mark in the center images, the region around that is considered muscle; the differentiation between healthy and injured is performed based on the echotexture features)
Song is directed toward, “ provide systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Torres is directed toward “to analyze the most frequent echotexture findings of patients with soleus muscle injury, located in the central intramuscular tendon (IMT), and healthy people to determine whether they behave differently and to propose an ultrasound (US)-based classification (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand diagnosis and medical reporting are often done based on baseline data; said differently, detecting abnormal patterns is often performed based on comparing a patient to a normal data set to detect differences. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to ensure deviations of a patient in question from a healthy patient can be detected for diagnosis.
Regarding dependent claim 24, the rejection of claim 23 is incorporated herein. Additionally, Torres discloses wherein the operations further comprise color coding, based on the identified musculoskeletal area, a portion of one or more images of the plurality of images that corresponds to the musculoskeletal area, wherein different musculoskeletal areas are associated with different colors (Figure 2, “Middle panel: ultrasound image with marked lesion area (yellow line)” the unhealthy region of the musculoskeletal area is associated with the yellow marking).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand color coding an image for review by a user can aid in an understanding of the image itself; said differently, color coding can provide information as related to size, location, etc of a region upon review without a user having to perform that analysis themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Torres in order to increase reviewer efficiency and understanding.
Claim(s) 2-5 and 10-13 are rejected under 35 U.S.C. 103 as being unpatentable over Song further in view of Torres as applied to claims 1 and 9 respectively above, and further in view of WO 2020079696 to Spillinger (hereinafter Spillinger).
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, Song and Torres fails to explicitly disclose further comprising automatically selecting a subset of images from the plurality of images for the report.
However, Spillinger discloses further comprising automatically selecting a subset of images from the plurality of images for the report (page 19, “According to some embodiments, images captured in vivo may be received and a plurality of these images may be automatically (e.g. by a processor shown in FIG. 1 ) selected for display. In some embodiments, a subset of these selected images may be identified automatically and/or by a user and a case report or a report may be generated which includes only images from the identified subset of images (e.g., one or more or all of the images in the subset).”).
As noted above, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, Song is directed toward “Embodiments of the disclosure provide systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Spillinger is directed toward “Systems and methods may display and/or provide analysis of a number of selected images of a patient's gastrointestinal tract collected in-vivo by a swallowable capsule. Images may be displayed for review (e.g., as a study) and/or for further analysis by a user. A subset of images representing the stream of images and automatically selected according to a first selection method may be displayed (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention Song, Torres and Spillinger are directed toward similar methods of endeavor of medical image analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would be aware providing thousands of images to a reviewer would be overwhelming, time consuming, and unnecessary. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Spillinger in order to ensure a report can be reviewed quickly for diagnosis.
Regarding dependent claim 3, the rejection of claim 2 is incorporated herein. Additionally, Spillinger in the combination further discloses wherein the subset of images are selected using the artificial intelligence system (page 11, “ Each image selection method described herein may include one or more filters or selection or detection rules and the selection according to each method may be performed in one or more stages. The selection or detection rules may be applied by utilizing algorithms, e.g., machine learning algorithms and deep learning algorithms in particular. ”).
It is well known to one of ordinary skill in the art before the effective filing date of the claimed invention artificial intelligence systems are faster and more accurate than other processing methods. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Spillinger in order to ensure a subset of images is determined accurately and efficiently.
Regarding dependent claim 4, the rejection of claim 2 is incorporated herein. Additionally, Song, Torres and Spillinger in the combination as a whole fails to explicitly disclose wherein the subset of images have a higher quality when compared to other images of the plurality of images. However, Spillinger does disclose at page 35, “The map screen or display of Fig. 3 may also include a graphical representation of, or an indication of, an estimated or determined cleansing level, e.g., a score for an automatically measured level of cleanliness of the respective segment, and/or an indication of the image quality during the capsule passage in that segment;” this cleansing value is read as a quality metric. Additionally, Spillinger discloses at page 11, “ Each image selection method described herein may include one or more filters or selection or detection rules and the selection according to each method may be performed in one or more stages. The selection or detection rules may be applied by utilizing algorithms, e.g., machine learning algorithms and deep learning algorithms in particular.” It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that images of high quality are best suited for diagnosis; said differently, if a low quality image is reviewed by a clinician, the clinician may determine an inaccurate diagnosis because the image quality it so poor. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Song, Torres and Spillinger in order to ensure the best images for review are output for diagnosis, so that accurate diagnosis can be made.
Regarding dependent claim 5, the rejection of claim 2 is incorporated herein. Additionally, Spillinger in the combination further discloses wherein the subset of images include one or more color codes that are associated with the at least one of the pathology and the musculoskeletal area (page 27, “ "Heat" may refer to colors used to convey information on the map, such as colors assigned to lines signifying certain images along the bar. Different colors may represent, for example, different type of images, such as images of a first or second level. Such a heat map or bar may include lines indicating the segmentation of the Gl portion (e.g., colored in a different color) etc. Colored sections of the map may provide certain information about those sections;” page 37, “Each selected image may be displayed with an indication of the identified area of interest within the image, e.g., by coloring the area of interest in a specific color.”).
It is well known to one of ordinary skill in the art before the effective filing date of the claimed invention that color coding images for review makes it easier and faster for a reviewer to understand. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Spillinger in order to ensure a subset of images are reviewed accurately and efficiently.
Regarding dependent claim 10, the rejection of claim 9 is incorporated herein. Additionally, Song and Torres fails to explicitly disclose further comprising instructions for selecting a subset of images from the plurality of images for the report.
However, Spillinger discloses further comprising instructions for selecting a subset of images from the plurality of images for the report (page 19, “According to some embodiments, images captured in vivo may be received and a plurality of these images may be automatically (e.g. by a processor shown in FIG. 1 ) selected for display. In some embodiments, a subset of these selected images may be identified automatically and/or by a user and a case report or a report may be generated which includes only images from the identified subset of images (e.g., one or more or all of the images in the subset).”).
As noted above, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, Song is directed toward “Embodiments of the disclosure provide systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Spillinger is directed toward “Systems and methods may display and/or provide analysis of a number of selected images of a patient's gastrointestinal tract collected in-vivo by a swallowable capsule. Images may be displayed for review (e.g., as a study) and/or for further analysis by a user. A subset of images representing the stream of images and automatically selected according to a first selection method may be displayed (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention Song, Torres and Spillinger are directed toward similar methods of endeavor of medical image analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would be aware providing thousands of images to a reviewer would be overwhelming, time consuming, and unnecessary. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Spillinger in order to ensure a report can be reviewed quickly for diagnosis.
Regarding dependent claim 11, the rejection of claim 10 is incorporated herein. Additionally, Spillinger in the combination further discloses wherein the subset of images are selected using the artificial intelligence system (page 11, “ Each image selection method described herein may include one or more filters or selection or detection rules and the selection according to each method may be performed in one or more stages. The selection or detection rules may be applied by utilizing algorithms, e.g., machine learning algorithms and deep learning algorithms in particular. ”).
It is well known to one of ordinary skill in the art before the effective filing date of the claimed invention artificial intelligence systems are faster and more accurate than other processing methods. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Spillinger in order to ensure a subset of images is determined accurately and efficiently.
Regarding dependent claim 12, the rejection of claim 10 is incorporated herein. Additionally, Song, Torres and Spillinger in the combination as a whole fails to explicitly disclose wherein the subset of images have a higher quality when compared to other images of the plurality of images. However, Spillinger does disclose at page 35, “The map screen or display of Fig. 3 may also include a graphical representation of, or an indication of, an estimated or determined cleansing level, e.g., a score for an automatically measured level of cleanliness of the respective segment, and/or an indication of the image quality during the capsule passage in that segment;” this cleansing value is read as a quality metric. Additionally, Spillinger discloses at page 11, “ Each image selection method described herein may include one or more filters or selection or detection rules and the selection according to each method may be performed in one or more stages. The selection or detection rules may be applied by utilizing algorithms, e.g., machine learning algorithms and deep learning algorithms in particular.” It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that images of high quality are best suited for diagnosis; said differently, if a low quality image is reviewed by a clinician, the clinician may determine an inaccurate diagnosis because the image quality it so poor. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Song, Torres and Spillinger in order to ensure the best images for review are output for diagnosis, so that accurate diagnosis can be made.
Regarding dependent claim 13, the rejection of claim 10 is incorporated herein. Additionally, Spillinger in the combination further discloses further comprising instructions for adding one or more color codes that are associated with the at least one of the pathology and the musculoskeletal area (page 27, “ "Heat" may refer to colors used to convey information on the map, such as colors assigned to lines signifying certain images along the bar. Different colors may represent, for example, different type of images, such as images of a first or second level. Such a heat map or bar may include lines indicating the segmentation of the Gl portion (e.g., colored in a different color) etc. Colored sections of the map may provide certain information about those sections;” page 37, “Each selected image may be displayed with an indication of the identified area of interest within the image, e.g., by coloring the area of interest in a specific color.”).
It is well known to one of ordinary skill in the art before the effective filing date of the claimed invention that color coding images for review makes it easier and faster for a reviewer to understand. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Spillinger in order to ensure a subset of images are reviewed accurately and efficiently.
Claim(s) 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Song and Torres further in view of Spillinger as applied to claims 5 and 13 respectively above, and further in view of U.S. Patent No. 11,676,701 to Carter et al. (hereinafter Carter).
Regarding dependent claim 6, the rejection of claim 5 is incorporated herein. Additionally, Song, Torres and Spillinger in the combination fails to explicitly disclose wherein an opacity of the one or more color codes is based on a determined severity of the at least one of the pathology and the musculoskeletal area.
However, Carter discloses wherein an opacity of the one or more color codes is based on a determined severity of the at least one of the pathology and the musculoskeletal area (column 10, line 48, “At block 406, for the given image region currently being processed, the medical provider system 102 may determine one or more bounding shape display parameters (such as color, opacity and/or shape type) based at least in part on a label within the metadata for the given region. The label may represent or specify a specific pathology or other classification previously determined by a machine learning model and assigned as a classification label to the given region. In some embodiments, for instance, different pathologies may be assigned different bounding shapes, colors or other display parameters, which may be configurable by a user. In one example, at least one display parameter determined at block 406 may be based on a confidence level determined by one or more models. For example, a specific color and/or opacity may be assigned to the bounding region based on its confidence score, as will be further discussed below.” Different pathologies are read as different severityies – i.e. cancer stage I vs stage II)).
As noted above, Song, Torres and Spillinger are directed toward medical image analysis. Further, Carter is directed toward “Systems and methods are provided for automatically marking locations within a radiograph of one or more dental pathologies, anatomies, anomalies or other conditions determined by automated image analysis of the radiograph by a number of different machine learning models (abstract).” As can be seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song, Torres, Spillinger and Carter are directed toward similar methods of endeavor of medical image analysis. Further, Carter allows for correlating the display to different diagnosis. One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand that outputting severity of a disease aids a user in understanding an overall output. Disease of lower severity may have different treatment than those of higher severity. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Carter to ensure a user has context for how severe a diagnosis of a patient is to inform further treatment.
Regarding dependent claim 14, the rejection of claim 13 is incorporated herein. Additionally, Song, Torres and Spillinger in the combination fails to explicitly disclose wherein an opacity of the one or more color codes is based on a determined severity of the at least one of the pathology and the musculoskeletal area.
However, Carter discloses wherein an opacity of the one or more color codes is based on a determined severity of the at least one of the pathology and the musculoskeletal area (column 10, line 48, “At block 406, for the given image region currently being processed, the medical provider system 102 may determine one or more bounding shape display parameters (such as color, opacity and/or shape type) based at least in part on a label within the metadata for the given region. The label may represent or specify a specific pathology or other classification previously determined by a machine learning model and assigned as a classification label to the given region. In some embodiments, for instance, different pathologies may be assigned different bounding shapes, colors or other display parameters, which may be configurable by a user. In one example, at least one display parameter determined at block 406 may be based on a confidence level determined by one or more models. For example, a specific color and/or opacity may be assigned to the bounding region based on its confidence score, as will be further discussed below.” Different pathologies are read as different severityies – i.e. cancer stage I vs stage II).
As noted above, Song, Torres and Spillinger are directed toward medical image analysis. Further, Carter is directed toward “Systems and methods are provided for automatically marking locations within a radiograph of one or more dental pathologies, anatomies, anomalies or other conditions determined by automated image analysis of the radiograph by a number of different machine learning models (abstract).” As can be seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song, Torres, Spillinger and Carter are directed toward similar methods of endeavor of medical image analysis. Further, Carter allows for correlating the display to different diagnosis. One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand that outputting severity of a disease aids a user in understanding an overall output. Disease of lower severity may have different treatment than those of higher severity. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Carter to ensure a user has context for how severe a diagnosis of a patient is to inform further treatment.
Claim(s) 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Song and Torres as applied to claims 1 and 9 respectively above, and further in view of Carter.
Regarding dependent claim 8, the rejection of claim 1 is incorporated herein. Additionally, Song and Torres fails to explicitly disclose further comprising:
generating a confidence threshold associated with the at least one of the pathology and the musculoskeletal area; and
providing the confidence threshold in the report.
However, Carter discloses further comprising:
generating a confidence threshold associated with the at least one of the pathology and the musculoskeletal area (column 12, line 53, “For example, a green bounding box may indicate a high confidence score (falling above a first threshold), gold may indicate a medium confidence score (falling above a second threshold) and red may indicate a low confidence score (falling above a third threshold). In other embodiments different shapes, line styles or other visual differences may be used to distinguish confidence scores instead of or in addition to color differences.”); and
providing the confidence threshold in the report (column 12, line 53, “For example, a green bounding box may indicate a high confidence score (falling above a first threshold), gold may indicate a medium confidence score (falling above a second threshold) and red may indicate a low confidence score (falling above a third threshold). In other embodiments different shapes, line styles or other visual differences may be used to distinguish confidence scores instead of or in addition to color differences;” outputting the colors correlated to the threshold is read as providing the threshold in the report).
As noted above, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, Song is directed toward “ systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Carter is directed toward “Systems and methods are provided for automatically marking locations within a radiograph of one or more dental pathologies, anatomies, anomalies or other conditions determined by automated image analysis of the radiograph by a number of different machine learning models (abstract).” As can be seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song, Torres and Carter are directed toward similar methods of endeavor of medical image analysis. Further, Carter allows for correlating the display to confidence determinations. One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand that outputting confidence information aids a user in determining how much one should rely on a specific output; said differently, if the system has low confidence in the output, the user may not want to rely heavily on that determination. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Carter to ensure a user has context for how confident a system is, to further inform their own decision making.
Regarding dependent claim 16, the rejection of claim 9 is incorporated herein. Additionally, Song and Torres fails to explicitly disclose further comprising:
generating a confidence threshold associated with the at least one of the pathology and the musculoskeletal area; and
providing the confidence threshold in the report.
However, Carter discloses further comprising:
generating a confidence threshold associated with the at least one of the pathology and the musculoskeletal area (column 12, line 53, “For example, a green bounding box may indicate a high confidence score (falling above a first threshold), gold may indicate a medium confidence score (falling above a second threshold) and red may indicate a low confidence score (falling above a third threshold). In other embodiments different shapes, line styles or other visual differences may be used to distinguish confidence scores instead of or in addition to color differences.”); and
providing the confidence threshold in the report (column 12, line 53, “For example, a green bounding box may indicate a high confidence score (falling above a first threshold), gold may indicate a medium confidence score (falling above a second threshold) and red may indicate a low confidence score (falling above a third threshold). In other embodiments different shapes, line styles or other visual differences may be used to distinguish confidence scores instead of or in addition to color differences;” outputting the colors correlated to the threshold is read as providing the threshold in the report).
As noted above, Song and Torres are directed towed similar methods of endeavor of medical image analysis. Further, Song is directed toward “ systems and methods for generating a diagnosis report based on a medical image of a patient (abstract).” Carter is directed toward “Systems and methods are provided for automatically marking locations within a radiograph of one or more dental pathologies, anatomies, anomalies or other conditions determined by automated image analysis of the radiograph by a number of different machine learning models (abstract).” As can be seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Song, Torres and Carter are directed toward similar methods of endeavor of medical image analysis. Further, Carter allows for correlating the display to confidence determinations. One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand that outputting confidence information aids a user in determining how much one should rely on a specific output; said differently, if the system has low confidence in the output, the user may not want to rely heavily on that determination. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Carter to ensure a user has context for how confident a system is, to further inform their own decision making.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Patent No. 11,246,527 to Stavros et al. discloses, “Systems, methods and computer program products are provided for reading and scoring ultrasound and/or optoacoustic (US/OA) images that include at least one of OA images or US images acquired in connection with an examination for a region of interest (ROI) (abstract).”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Primary Examiner, Art Unit 2661
Prosecution Insights