DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Receipt is acknowledged that application claims priority to foreign application with application number JP2023-217180 dated 12/22/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 12/18/2024 has been considered and placed in the application file.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 14 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 14 recites the limitation, "wherein the one or more processors replace a value of a degree of certainty of a pixel constituting the degree-of-certainty distribution information with a maximum value of a degree of certainty in a local region located within a specified range from the pre-specified pixel”. Here the phrase “the pre-specified pixel” has a lack of antecedent basis. Therefore, the scope of the limitation is unclear.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-6, and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable and obvious over US Patent Application Publication US 2021/0056690 A1, (FUTAMURA et al.) (hereinafter “Futamura”) in view of US Patent Application Publication US 2023/0015933 A1, (AOYAGI et al.) (hereinafter “Aoyagi”).
Regarding claim 1, Futamura teaches an information processing apparatus comprising: (Futamura “[Abstract] A medical information processing apparatus…”)
one or more processors; and (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
one or more memories that store instructions to be executed by the one or more processors, (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
wherein the one or more processors acquire a medical image, (Futamura “[0055] The image server 3 is , for example , a server of a PACS ( Picture Archiving and Communication System ) , and associates and stores , in a database , each medical image output from the modality 1 with the patient information ( patient ID , name , birth date , age , sex , height , weight , etc. ) , the examination information ( examination ID , examination date and time , modality type , examination site , client department , examination purpose , etc. ) , the image ID of the medical image , and the detection result information and the display information of lesion - detected regions output from the medical information processing apparatus 2.”)
…a degree of certainty representing at least any of presence or absence of a disease or a degree of the disease for each pixel of the medical image, (Futamura Figs. 6 and 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction ( differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
create degree-of-certainty distribution information for visualizing a distribution of the degree of certainty for each detected disease, (Futamura Figs. 6 and 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
acquire a threshold value to be applied to the degree of certainty, and (Futamura “[0061] …as shown in FIG. 6, the controller 21 binarizes the heatmap information by using a predetermined threshold value, and identifies a region (s) equal to or larger than the threshold value (region filled with black in FIG . 6) as a lesion - detected region(s).”)
perform control of superimposing at least any of a first color or a first opacity on the medical image for a pixel of a first image having the degree of certainty of the same value as the threshold value and displaying the superimposed image on a display device, for any disease (Futamura “[0061] …as shown in FIG. 6, the controller 21 binarizes the heatmap information by using a predetermined threshold value, and identifies a region (s) equal to or larger than the threshold value (region filled with black in FIG. 6) as a lesion - detected region(s).”; “[0090] In Step S4, by processing the detection result information, the controller 21 generates the display information of the lesion - detected regions (heatmap display information of each lesion - detected region and character information indicating the type of the lesion in each lesion detected region) that is superimposed on the medical image. The heatmap display information is, for example, information colored according to the values of the certainty degrees.”; “[0105] FIG . 8A shows an example of an image in which the heatmap information of multiple types of lesions detected in a present medical image of a patient is colored, and superimposed and displayed as it is on the present medical image…”; “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit , a display , a storage and a communication unit , and reads out a medical image and its display information of lesion - detected regions from the image server 3 by making a request to the image server 3 , and displays these for interpretation.”)
However, Futamura is silent about using a multi-disease detection model for detecting a plurality of disease regions from the medical image to acquire, for each disease.
Aoyagi teaches use a multi-disease detection model for detecting a plurality of disease regions from the medical image to acquire, for each disease, (Aoyagi Figs. 2, 6, and 8, “[0030] …a trained model for diagnosing disease is constructed by machine learning with the use of training dataset including medical images. On the other hand, in the inference phase, new medical images are inputted to the trained model constructed in the training phase to obtain inferred disease information.”; “[0072] …when the disease information inferred by the trained model is the disease name “cardiomegalies” and the diseased site is the entirety of the enlarged heart, the disease name “cardiomegalia” may be displayed next to the 2D virtual projection image, and a square frame indicating the diseased site may be superimposed on the 2D virtual projection image.”; “[0134] …when the inferred disease is pulmonary nodule or cardiomegalia, as illustrated in FIG. 14, imaging conditions for the detailed examination are set for the lung region and the cardiac region that are important for diagnosing the inferred disease…”)
Futamura and Aoyagi are analogous art as both of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura by using a multi-disease detection model for detecting a plurality of disease regions from the medical image to acquire, for each disease, as taught by Aoyagi and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced visualization of detected diseases in medical images.
Claim 16 is directed to an operation method of an information processing apparatus, the method being executed by a computer functioning as the information processing apparatus and comprising: and its steps are substantially similar to the scope and functions performed by the apparatus claim 1 and therefore claim 16 is also rejected with the same rationale as specified in the rejection of claim 1.
Claim 17 is directed to a non-transitory, computer-readable tangible recording medium which records thereon a program for causing, when read by a computer functioning as an information processing apparatus, the computer to implement: (Futamura “[0011] In order to achieve at least one of the objects , according to another aspect of the present disclosure , there is provided a non - transitory computer readable storage medium storing a program to cause a computer to…”) and its scope and functions are substantially similar to the scope and functions performed by the apparatus claim 1 and therefore claim 17 is also rejected with the same rationale as specified in the rejection of claim 1.
Regarding claim 3, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) classify a pixel located at a distance within a specified range from a pixel classified as the pixel of the first image, as the pixel of the first image (Futamura “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”; “[0069] The area of each lesion - detected region can be obtained from the number of pixels of the lesion - detected region…The length (dimension) in the longer axis direction of each lesion - detected region can be obtained from the number of pixels of the maximum width of the lesion - detected region, for example.”).
Regarding claim 4, Futamura teaches wherein the one or more processors perform control of, (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) for a pixel of a second image of which the degree of certainty exceeds the degree of certainty in the pixel of the first image, (Futamura Figs. 8A-8B, “[0090] …The heatmap display information is, for example, information colored according to the values of the certainty degrees.”). The Examiner would like to note that in Fig. 8A (Futamura) there is a new type of lesion detected in the image that wasn’t in the previous image (in Fig. 8B)—which means that the “degree of certainty” for a disease has exceeded in the second image than it did in the first image. superimposing and displaying at least any of a second color corresponding to a maximum value of the degree of certainty or a second opacity corresponding to the maximum value of the degree of certainty on the medical image, for any disease (Futamura Figs. 8A-8B, “[0116] The representative point may be the centroid of a lesion-detected region, a point where the certainty degree is the maximum value, or the centroid of a region where the certainty degree(s) is a predetermined value or larger…”; “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction) and slopes in the y direction (differences between pixel values of pixels adjacent to one another in the y direction) of the heatmap information, and regarding the maximum value (“ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”). The Examiner would like to note that the image in Fig. 8B shows two types of diseases and a varying range of color depending on the degree of certainty.
Regarding claim 5, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) set a third opacity lower than the first opacity for the pixel of the second image in a case in which a pixel adjacent to the pixel of the second image is classified as the pixel of the first image (Futamura Figs. 8A-8B, “[0116] The representative point may be the centroid of a lesion-detected region, a point where the certainty degree is the maximum value, or the centroid of a region where the certainty degree(s) is a predetermined value or larger…”; “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction) and slopes in the y direction (differences between pixel values of pixels adjacent to one another in the y direction) of the heatmap information, and regarding the maximum value (“ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”). The Examiner would like to note that the image in Fig. 8B shows two types of diseases and a varying range of opacity depending on the degree of certainty.
Regarding claim 6, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) perform control of causing the medical image to be displayed with a non-superimposed color and a non-superimposed opacity for a pixel of a third image of which the degree of certainty is less than the degree of certainty in the pixel of the first image, for any disease (Futamura Figs. 9C, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction) and slopes in the y direction (differences between pixel values of pixels adjacent to one another in the y direction) of the heatmap information, and regarding the maximum value (“ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”; “[0110] …the detection result information has been processed with the processing method IDs of 004 and 001 (switch images in descending order of priority+make character attribute differ)…This allows an interpreter to first interpret new lesion-detected regions, which are not present in a past image(s), with focus thereon…”). The Examiner would like to note that Fig. 9C (Futamura) depicts two images with different lesions detected. The regions where the lesions are detected have superimposed color and opacity, whereas the regions where no lesions are detected have non-superimposed color and non-superimposed opacity.
Regarding claim 14, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) replace a value of a degree of certainty of a pixel constituting the degree-of-certainty distribution information with a maximum value of a degree of certainty in a local region located within a specified range from the pre-specified pixel (Futamura Fig. 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”; “[0069] The area of each lesion - detected region can be obtained from the number of pixels of the lesion - detected region…The length (dimension) in the longer axis direction of each lesion - detected region can be obtained from the number of pixels of the maximum width of the lesion - detected region, for example.”)
Regarding claim 15, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) receive an input of a user for making a change to control of displaying the degree-of-certainty distribution information (Futamura Fig. 6, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”)
Claims 2, 7, 9, and 18 are rejected under 35 U.S.C. 103 as being unpatentable and obvious over Futamura and Aoyagi as applied to claims 1, 3-6, and 14-17 above, and further in view of US Patent Application Publication US 2022/0285031 A1, (SATO et al.) (hereinafter “Sato”).
Regarding claim 2, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
and classify a pixel having the degree of certainty within the threshold value range as the pixel of the first image (Futamura “[0061] …as shown in FIG. 6, the controller 21 binarizes the heatmap information by using a predetermined threshold value, and identifies a region (s) equal to or larger than the threshold value (region filled with black in FIG. 6) as a lesion - detected region(s).”; “[0090] In Step S4, by processing the detection result information, the controller 21 generates the display information of the lesion - detected regions (heatmap display information of each lesion - detected region and character information indicating the type of the lesion in each lesion detected region) that is superimposed on the medical image. The heatmap display information is, for example, information colored according to the values of the certainty degrees.”; “[0105] FIG . 8A shows an example of an image in which the heatmap information of multiple types of lesions detected in a present medical image of a patient is colored, and superimposed and displayed as it is on the present medical image…”; “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit , a display , a storage and a communication unit , and reads out a medical image and its display information of lesion - detected regions from the image server 3 by making a request to the image server 3 , and displays these for interpretation.”)
However, Futamura and Aoyagi are silent about acquire a threshold value range having a pre-specified upper limit value exceeding the threshold value and a pre-specified lower limit value less than the threshold value.
Sato teaches acquire a threshold value range having a pre-specified upper limit value exceeding the threshold value and a pre-specified lower limit value less than the threshold value, (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
Futamura, Aoyagi, and Sato are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by acquiring a threshold value range having a pre-specified upper limit value exceeding the threshold value and a pre-specified lower limit value less than the threshold value as taught by Sato and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Regarding claim 7, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
acquire a maximum value of the degree of certainty of each disease, (Futamura Figs. 8A-8B, “[0116] The representative point may be the centroid of a lesion-detected region, a point where the certainty degree is the maximum value, or the centroid of a region where the certainty degree(s) is a predetermined value or larger…”)
perform control of displaying text representing a disease name (Futamura Figs. 8A-8B, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”). The Examiner would like to note that figures 8A and 8B (Futamura) display the text information of the disease name.
and the maximum value of the degree of certainty in a case in which the maximum value of the degree of certainty exceeds the threshold value (Futamura Figs. 8A-8B, “[0116] The representative point may be the centroid of a lesion-detected region, a point where the certainty degree is the maximum value, or the centroid of a region where the certainty degree(s) is a predetermined value or larger…”; “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction) and slopes in the y direction (differences between pixel values of pixels adjacent to one another in the y direction) of the heatmap information, and regarding the maximum value (“ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”). The Examiner would like to note that the image in Fig. 8B shows two types of diseases and a varying range of color depending on the degree of certainty.
However, Futamura and Aoyagi are silent about determine whether or not the maximum value of the degree of certainty exceeds the threshold value for each disease.
Sato teaches determine whether or not the maximum value of the degree of certainty exceeds the threshold value for each disease, and (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
Futamura, Aoyagi, and Sato are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by determining whether or not the maximum value of the degree of certainty exceeds the threshold value for each disease as taught by Sato and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Regarding claim 9, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
receive an input of a user who designates a pixel in the medical image, and (Futamura “[0044] The parameter IDs can each be specified, for example, by a user(s) through the operation unit 24…”; “[0078] …the controller 21 determines the priority degree of each lesion-detected region on the basis of whether the lesion-detected region is in an area(s) specified by the user (e.g. doctor in charge). More specifically, the controller 21 compares the position information of each lesion-detected region (coordinate information of each region extracted by binarizing the heatmap information by using a predetermined threshold value) with a user-specified area(s), which has been specified through the operation unit 24, on the medial image displayed on the display 26, and determines lesion-detected regions located in the user-specified area…”)
perform control of, in a case in which the degree of certainty of each disease in the designated pixel…displaying a value of the degree of certainty (Futamura Fig. 6, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”; “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”; “[0069] The area of each lesion - detected region can be obtained from the number of pixels of the lesion - detected region…The length (dimension) in the longer axis direction of each lesion - detected region can be obtained from the number of pixels of the maximum width of the lesion - detected region, for example.”)
However, Futamura and Aoyagi are silent about exceeds the threshold value.
Sato teaches exceeds the threshold value (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
Futamura, Aoyagi, and Sato are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by exceeding the threshold value as taught by Sato and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Regarding claim 18, Futamura teaches an information processing apparatus comprising: (Futamura “[Abstract] A medical information processing apparatus…”)
one or more processors; and (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
one or more memories that store instructions to be executed by the one or more processors, (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
wherein the one or more processors acquire a medical image, (Futamura “[0055] The image server 3 is , for example , a server of a PACS ( Picture Archiving and Communication System ) , and associates and stores , in a database , each medical image output from the modality 1 with the patient information ( patient ID , name , birth date , age , sex , height , weight , etc. ) , the examination information ( examination ID , examination date and time , modality type , examination site , client department , examination purpose , etc. ) , the image ID of the medical image , and the detection result information and the display information of lesion - detected regions output from the medical information processing apparatus 2.”)
…a degree of certainty representing at least any of presence or absence of a disease or a degree of the disease for each pixel of the medical image, (Futamura Figs. 6 and 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction ( differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
acquire a threshold value to be applied to the degree of certainty, (Futamura “[0061] …as shown in FIG. 6, the controller 21 binarizes the heatmap information by using a predetermined threshold value, and identifies a region (s) equal to or larger than the threshold value (region filled with black in FIG . 6) as a lesion - detected region(s).”)
acquire information on a region for each anatomical structure from the medical image, (Futamura Figs. 6 & 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”; “[0069] The area of each lesion - detected region can be obtained from the number of pixels of the lesion - detected region…The length (dimension) in the longer axis direction of each lesion - detected region can be obtained from the number of pixels of the maximum width of the lesion - detected region, for example.”; “[0113] As shown in FIG. 10, when a medical image G is input to the medical information processing apparatus 2, the detector 25 detects multiple types of lesions in the medical image G and outputs the detection result information. As described above, the detection result information includes, for each lesion type, the heatmap information and the supplementary information including a lesion type…”; “[0115] …The checking method is, in the case of chest, selecting one from the upper lung field, the middle lung field and the lower lung field into which a lung field is divided in advance, thereby selecting a region where a lesion is located…”)
acquire a maximum value of the degree of certainty of each disease, for the region for each anatomical structure, and…and the maximum value of the degree of certainty in a case in which the maximum value of the degree of certainty of each disease in the region for each anatomical structure (Futamura Figs. 8A-8B, “[0116] The representative point may be the centroid of a lesion-detected region, a point where the certainty degree is the maximum value, or the centroid of a region where the certainty degree(s) is a predetermined value or larger…”)
However, Futamura is silent about using a multi-disease detection model for detecting a plurality of disease regions from the medical image to acquire, for each disease.
Aoyagi teaches use a multi-disease detection model for detecting a plurality of disease regions from the medical image to acquire, for each disease, (Aoyagi Figs. 2, 6, and 8, “[0030] …a trained model for diagnosing disease is constructed by machine learning with the use of training dataset including medical images. On the other hand, in the inference phase, new medical images are inputted to the trained model constructed in the training phase to obtain inferred disease information.”; “[0072] …when the disease information inferred by the trained model is the disease name “cardiomegalies” and the diseased site is the entirety of the enlarged heart, the disease name “cardiomegalia” may be displayed next to the 2D virtual projection image, and a square frame indicating the diseased site may be superimposed on the 2D virtual projection image.”; “[0134] …when the inferred disease is pulmonary nodule or cardiomegalia, as illustrated in FIG. 14, imaging conditions for the detailed examination are set for the lung region and the cardiac region that are important for diagnosing the inferred disease…”)
However, Futamura and Aoyagi are silent about storing a combination of an anatomical structure name, a disease name…exceeds the threshold value.
Sato teaches store a combination of an anatomical structure name, a disease name…exceeds the threshold value (Sato Fig. 3, “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
Futamura, Aoyagi, and Sato are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura by using a multi-disease detection model for detecting a plurality of disease regions from the medical image to acquire, for each disease as taught by Aoyagi and by storing a combination of an anatomical structure name, a disease name…exceeds the threshold value as taught by Sato and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Claims 8 and 10-13 are rejected under 35 U.S.C. 103 as being unpatentable and obvious over Futamura, Aoyagi, and Sato as applied to claims 2, 7, 9, and 18 above, and further in view of US Patent Application Publication US 2020/0065963 A1, (Aoyagi) (hereinafter “Aoyagi 2”).
Regarding claim 8, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
…including a plurality of consecutive pixels of which a value of the degree of certainty of each disease (Futamura Figs. 6 and 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction ( differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
determine whether or not a maximum value of the degree of certainty (Futamura Figs. 8A-8B, “[0116] The representative point may be the centroid of a lesion-detected region, a point where the certainty degree is the maximum value, or the centroid of a region where the certainty degree(s) is a predetermined value or larger…”)
perform control of displaying the maximum value of the degree of certainty of each disease in the designated connected region (Futamura Fig. 6, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”; “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction (differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”; “[0069] The area of each lesion - detected region can be obtained from the number of pixels of the lesion - detected region…The length (dimension) in the longer axis direction of each lesion - detected region can be obtained from the number of pixels of the maximum width of the lesion - detected region, for example.”)
However, Futamura and Aoyagi are silent about exceeds the threshold value…exceeds the threshold value for each connected region of each disease.
Sato teaches …exceeds the threshold value…exceeds the threshold value for each connected region of each disease, (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
However, Futamura, Aoyagi, and Sato are silent about execute labeling processing on one or more connected regions…receive an input of a user who designates the connected region.
Aoyagi 2 teaches execute labeling processing on one or more connected regions (Aoyagi 2, Fig. 7, “[0063] …in FIG. 7, the upper part of the right lung is classified into the type “B” indicative of the ground-glass opacities pattern, the remaining regions of the right lung is classified into the type “A” indicative of the normal pattern, while a large part of the left lung is classified into the type “C” indicative of the reticular and linear opacities pattern.”)
receive an input of a user who designates the connected region, and (Aoyagi 2 “[0029] …input interface circuit 10 is an interface circuit for inputting data via a storage medium such as an optical disk and/or a USB memory and for inputting data via a wired or wireless network or a special-purpose or general-purpose communication line. The medical image processing apparatus 100 of the first embodiment acquires the first and second images imaged by the modality 510 such as the X-ray CT apparatus 511 or the first and second images stored in the image server, via the input interface circuit 10.”; “[0032] The input device 40 includes various devices for an operator to input various types of information and data, and is configured of a mouse, a keyboard, a trackball, and a touch panel, for example.”; “[0098] …a user uses the input device 40 such as a mouse and/or a keyboard for designating a disease cause...”)
Futamura, Aoyagi, Sato, and Aoyagi 2 are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by exceeds the threshold value…exceeds the threshold value for each connected region of each disease as taught by Sato and by execute labeling processing on one or more connected regions…receive an input of a user who designates the connected region as taught by Aoyagi 2 and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Regarding claim 10, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
…including a plurality of consecutive pixels of which a value of the degree of certainty of each disease… (Futamura Figs. 6 and 7, “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction ( differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
However, Futamura and Aoyagi are silent about exceeds the threshold value.
Sato teaches exceeds the threshold value, and (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
However, Futamura, Aoyagi, and Sato are silent about execute labeling processing on one or more connected regions…perform control of displaying at least any of a disease name or the degree of certainty of the connected region for each connected region of each disease.
Aoyagi 2 teaches execute labeling processing on one or more connected regions… (Aoyagi 2, Fig. 7, “[0063] …in FIG. 7, the upper part of the right lung is classified into the type “B” indicative of the ground-glass opacities pattern, the remaining regions of the right lung is classified into the type “A” indicative of the normal pattern, while a large part of the left lung is classified into the type “C” indicative of the reticular and linear opacities pattern.”)
perform control of displaying at least any of a disease name or the degree of certainty of the connected region for each connected region of each disease (Aoyagi 2 “[0032] The input device 40 includes various devices for an operator to input various types of information and data, and is configured of a mouse, a keyboard, a trackball, and a touch panel, for example.”; “[0098] …a user uses the input device 40 such as a mouse and/or a keyboard for designating a disease cause...”; “[0041] The display control function 25 causes the display 50 to display the generated disease - state - change map , for example , in response to a user's instruction.”)
Futamura, Aoyagi, Sato, and Aoyagi 2 are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by exceeds the threshold value as taught by Sato and by execute labeling processing on one or more connected regions…perform control of displaying at least any of a disease name or the degree of certainty of the connected region for each connected region of each disease as taught by Aoyagi 2 and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Regarding claim 11, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”) perform control of displaying at least any of the disease name for each connected region or the degree of certainty of the connected region outside a detection target region in the medical image (Futamura Figs. 8A-8B, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”)
Regarding claim 12, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
…including a plurality of consecutive pixels of which the degree of certainty of each disease… (Futamura “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction ( differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
perform control of displaying all the degrees of certainty of each disease for each connected region (Futamura Figs. 6 and 7, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”)
However, Futamura and Aoyagi are silent about execute labeling processing on one or more connected regions.
Aoyagi 2 teaches execute labeling processing on one or more connected regions (Aoyagi 2, Fig. 7, “[0063] …in FIG. 7, the upper part of the right lung is classified into the type “B” indicative of the ground-glass opacities pattern, the remaining regions of the right lung is classified into the type “A” indicative of the normal pattern, while a large part of the left lung is classified into the type “C” indicative of the reticular and linear opacities pattern.”)
However, Futamura, Aoyagi, and Aoyagi 2 are silent about exceeds the threshold value.
Sato teaches …exceeds the threshold value, and… (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
Futamura, Aoyagi, Sato, and Aoyagi 2 are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by exceeds the threshold value as taught by Sato and by execute labeling processing on one or more connected regions as taught by Aoyagi 2 and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Regarding claim 13, Futamura teaches wherein the one or more processors (Futamura “[0034] As shown in FIG. 2, the medical information processing apparatus 2 includes a controller 21 (hardware processor)…”; “[0035] The controller 21 includes a CPU ( Central Processing Unit ) and a RAM ( Random Access Memory ) , and comprehensively controls operation of each component of the medical information processing apparatus 2…”)
…including a plurality of consecutive pixels of which the degree of certainty of each disease… (Futamura “[0072] The gradient of certainty degrees of the lesion in each lesion - detected region can be obtained, for example, as shown in FIG . 7, by calculating slopes in the x direction ( differences between pixel values of pixels adjacent to one another in the x direction ) and slopes in the y direction ( differences between pixel values of pixels adjacent to one another in the y direction ) of the heatmap information , and regarding the maximum value ( “ 50 ” in FIG . 7) of the absolute values of the slopes as a representative value of the slopes of certainty degrees…”; “[0122] …the digital data represents the heatmap information of one type of lesion. However, each bit in a bit string of each pixel may be made to have meaning, and one digital data may represent the heatmap information of multiple types of lesions…”)
perform control of displaying text information… (Futamura Figs. 8A-8B, “[0057] The interpretation terminal 4 is a computer apparatus that includes a controller, an operation unit, a display, a storage and a communication unit, and reads out a medical image and its display information of lesion-detected regions from the image server 3 by making a request to the image server 3, and displays these for interpretation.”). The Examiner would like to note that figures 8A and 8B (Futamura) display the text information of the disease name.
However, Futamura and Aoyagi are silent about execute labeling processing on one or more connected regions…obtained by integrating a plurality of disease names in an overlapping portion of each disease in a case in which the overlapping portion.
Aoyagi 2 teaches execute labeling processing on one or more connected regions… (Aoyagi 2, Fig. 7, “[0063] …in FIG. 7, the upper part of the right lung is classified into the type “B” indicative of the ground-glass opacities pattern, the remaining regions of the right lung is classified into the type “A” indicative of the normal pattern, while a large part of the left lung is classified into the type “C” indicative of the reticular and linear opacities pattern.”)
…obtained by integrating a plurality of disease names in an overlapping portion of each disease in a case in which the overlapping portion… (Aoyagi 2, Figs. 4 and 7, “[0050] As for the left lung (the lung depicted on the right side in the first and second images in FIG. 4), though a texture pattern of a type different from that of the right lung is generated over a wide range in the first image, the diseased region corresponding to this type of texture pattern is smaller in the second image. Further, in the second image, in the lower part of the left lung, a texture pattern of a type different from the above two types is generated. This means that the region in the recovery direction and the region in the exacerbation direction are mixed in the left lung.”; “[0064] …in FIG. 7, most of the right lung is classified into the type “B” indicative of the ground-glass opacities pattern, while the left lung is classified into the type “C” indicative of the reticular and linear opacities pattern and the type “D” indicative of nodular opacities pattern except the peripheral regions classified into the type “A” indicative of the normal pattern.”)
However, Futamura, Aoyagi, and Aoyagi 2 are silent about …exceeds the threshold value, and…exceeds a specified value, for each connected region of each disease.
Sato teaches …exceeds the threshold value, and…exceeds a specified value, for each connected region of each disease (Sato “[0153] …the calculating function 197 is configured to display a histogram of the pixels in a partial region, for example, so as to prompt the user to designate a threshold value (a reference value) for the normal region. FIG. 5 is a drawing illustrating an example of a screen for setting the threshold value for the pixel values. On a threshold value setting screen 15b, the display controlling function 203 is configured to display the histogram of the pixels in the partial region.”; “[0155] …even in the situations where it would be inappropriate to use the average value or the median value as the threshold value…”). The Examiner would like to note that in the histogram of Fig. 5 (in Sato), there is an “abnormal site” which contains pixel values that exceed the “reference value” (or threshold value)—these pixel values can be thought of as the “upper limit value”. There is also a “normal site” which contains pixel values that are less than the “reference value” (or threshold value)—these pixel values can be thought of as the “lower limit value”. The figure also shows three vertical lines to represent the threshold value range; the first line being before the threshold value, the second line being the threshold value, and the third line being after the threshold value.
Futamura, Aoyagi, Sato, and Aoyagi 2 are analogous art as all of them are related to medical imaging processing.
Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Futamura modified by Aoyagi by …exceeds the threshold value, and…exceeds a specified value, for each connected region of each disease as taught by Sato and by execute labeling processing on one or more connected regions…obtained by integrating a plurality of disease names in an overlapping portion of each disease in a case in which the overlapping portion as taught by Aoyagi 2 and use that within Futamura’s medical information processing apparatus.
The motivation for the above is for enhanced interpretation of detected diseases in medical images.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Patent Application Publication US20210192727A1 (Ward et al.) discloses using a deep learning model for identifying areas of interest in medical images.
US Patent Application Publication US 2018/0293465 A1, (KANADA) discloses a classification unit for types of disease for each pixel of a medical image.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMELIA VELAZQUEZ VALENCIA whose telephone number is (571)272-7418. The examiner can normally be reached M-F, 8:30AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A. Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.V.V/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Date: 06/10/2026
Prosecution Insights