DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1 and 7 – 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Aoyagi et al. (US 2015/0366500 A1; published on 12/24/2015) (hereinafter "Aoyagi").
Regarding claim 1, Aoyagi discloses an image processing device ("FIG. 1 is a diagram that illustrates an example of the configuration of a medical-image processing apparatus 100 …" [0016]) comprising
at least one memory configured to store instructions ("For example, the storage unit 140 is a hard disk, a semiconductor memory device, or the like, and it stores various types of information." [0022]), and
at least one processor ("The control unit 150 is, for example, an electronic circuit, such as a central processing unit (CPU) or a micro processing unit (MPU) ..." [0023]) configured to execute the instructions ("Moreover, all or any of various processing functions performed by each apparatus may be implemented by a CPU or a program that is analyzed and executed by the CPU …" [0075]) to:
acquire a medical image ("… the image acquiring unit 151 acquires medical image data from the medical-image diagnostic apparatus or the image storage apparatus via the communication unit 130 (Step S101)." [0042]);
generate, based on the medical image, plural options of a size of a region of interest included in the medical image ("In such a case, the dividing unit 153 performs erosion to extract multiple regions with different sizes in a step-by-step manner and repeats a subtraction operation in a stepwise fashion to subtract the region with the size that is one step smaller from the region with the size that is one step larger, thereby dividing the disease candidate region into three or more sub regions." [0032]; "Afterward, the dividing unit 153 divides the determined disease candidate region into multiple sub regions (Step S103), and the extracting unit 154 extracts a feature vector of each of the divided sub regions (Step S104)." [0043]); and
cause a display device to display information on the plural options ("The display unit 120 is a liquid crystal panel, or the like, and it displays various types of information. Specifically, the display unit 120 displays a graphical user interface (GUI) for receiving various operations from an operator, the estimation result of an estimation that is made during an operation by the control unit 150 that is described later, or the like." [0021]; see also [0044], the estimation results are associated with different sub regions).
Regarding claim 7, Aoyagi discloses all claim limitations, as applied in claim 1, and further discloses wherein the at least one processor is configured to execute the instructions ("Moreover, all or any of various processing functions performed by each apparatus may be implemented by a CPU or a program that is analyzed and executed by the CPU …" [0075]) to
make inference results for respective bit positions of coordinates of the region of interest in binary notation ("Then, the determining unit 152 stores, in the storage unit 140, a binary image in which the determined disease candidate region R1 and the other regions (background regions) are represented by using different values (e.g., the image in which the disease candidate region is “1” and the background region is “0”), as a mask image for limiting the processing range." [0026]), and
generate the plural options based on the inference results and plural sets of threshold value(s) to be compared with the inference results ("Specifically, the dividing unit 153 divides the disease candidate region R2 into a sub region R21, a sub region R22, and a sub region R23 as illustrated in FIG. 7 in accordance with the predetermined threshold that is previously set for CT values." [0054]).
Regarding claim 8, Aoyagi discloses all claim limitations, as applied in claim 1, and further discloses wherein the at least one processor is configured to execute the instructions ("Moreover, all or any of various processing functions performed by each apparatus may be implemented by a CPU or a program that is analyzed and executed by the CPU …" [0075]) to
extract features of the medical image ("With reference back to FIG. 1, the extracting unit 154 extracts each of the feature values that correspond to the sub regions, which are divided by the dividing unit 153, as the feature value of the disease candidate region." [0033]), and
make inference results for respective bit positions of coordinates of the region of interest in binary notation ("Then, the determining unit 152 stores, in the storage unit 140, a binary image in which the determined disease candidate region R1 and the other regions (background regions) are represented by using different values (e.g., the image in which the disease candidate region is “1” and the background region is “0”), as a mask image for limiting the processing range." [0026]), based on the features and machine learning models for the respective bit positions ("Then, the extracting unit 154 combines the extracted feature vector 1 and the feature vector 2 to extract the feature vector of the disease candidate region R1 and performs machine learning by using the extracted feature vector as an input." [0033]), and
wherein the machine learning models are trained, through machine learning, to learn a relation between features of an image and values at the respective bit positions of a region of interest in the image ("… and the estimation criterion can be generated by applying machine learning to training data (supervised image)." [0018]; "In such a case, for example, the extracting unit 154 first extracts the feature vector of the lung nodule region from each of the supervised images in which it is determined whether the extracted lung nodule is benign or malignant as described above, uses the feature vector to generate a classifier with which the estimating unit 155 classifies benignancy and malignancy, and stores it in the estimation-criterion storage unit 142." [0038]).
Regarding claim 9, Aoyagi discloses an image processing method executed by a computer ("FIG. 6 is a flowchart that illustrates the steps of an operation of the medical-image processing apparatus 100 according to the first embodiment." [0041]), comprising:
acquiring a medical image ("… the image acquiring unit 151 acquires medical image data from the medical-image diagnostic apparatus or the image storage apparatus via the communication unit 130 (Step S101)." [0042]);
generating, based on the medical image, plural options of a size of a region of interest included in the medical image ("In such a case, the dividing unit 153 performs erosion to extract multiple regions with different sizes in a step-by-step manner and repeats a subtraction operation in a stepwise fashion to subtract the region with the size that is one step smaller from the region with the size that is one step larger, thereby dividing the disease candidate region into three or more sub regions." [0032]; "Afterward, the dividing unit 153 divides the determined disease candidate region into multiple sub regions (Step S103), and the extracting unit 154 extracts a feature vector of each of the divided sub regions (Step S104)." [0043]); and
causing a display device to display information on the plural options ("The display unit 120 is a liquid crystal panel, or the like, and it displays various types of information. Specifically, the display unit 120 displays a graphical user interface (GUI) for receiving various operations from an operator, the estimation result of an estimation that is made during an operation by the control unit 150 that is described later, or the like." [0021]; see also [0044], the estimation results are associated with different sub regions).
Regarding claim 10, Aoyagi discloses a non-transitory computer readable storage medium storing a program executed by a computer ("For example, the storage unit 140 is a hard disk, a semiconductor memory device, or the like, and it stores various types of information." [0022]; "Moreover, all or any of various processing functions performed by each apparatus may be implemented by a CPU or a program that is analyzed and executed by the CPU …" [0075]), the program causing the computer to:
acquire a medical image ("… the image acquiring unit 151 acquires medical image data from the medical-image diagnostic apparatus or the image storage apparatus via the communication unit 130 (Step S101)." [0042]);
generate, based on the medical image, plural options of a size of a region of interest included in the medical image ("In such a case, the dividing unit 153 performs erosion to extract multiple regions with different sizes in a step-by-step manner and repeats a subtraction operation in a stepwise fashion to subtract the region with the size that is one step smaller from the region with the size that is one step larger, thereby dividing the disease candidate region into three or more sub regions." [0032]; "Afterward, the dividing unit 153 divides the determined disease candidate region into multiple sub regions (Step S103), and the extracting unit 154 extracts a feature vector of each of the divided sub regions (Step S104)." [0043]); and
cause a display device to display information on the plural options ("The display unit 120 is a liquid crystal panel, or the like, and it displays various types of information. Specifically, the display unit 120 displays a graphical user interface (GUI) for receiving various operations from an operator, the estimation result of an estimation that is made during an operation by the control unit 150 that is described later, or the like." [0021]; see also [0044], the estimation results are associated with different sub regions).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Aoyagi, as applied in claim 1, and further in view of Douglas et al. (US 2021/0294435 A1; published on 09/23/2021) (hereinafter "Douglas").
Regarding claim 2, Aoyagi teaches all claim limitations, as applied in claim 1, and Aoyagi further teaches wherein the at least one processor is configured to execute the instructions ("Moreover, all or any of various processing functions performed by each apparatus may be implemented by a CPU or a program that is analyzed and executed by the CPU …" [0075]) to
generate a frequency distribution of the plural options of the size of the region of interest ("For example, as illustrated in (A) of FIG. 5, the extracting unit 154 calculates the histogram of pixel values of the sub region R11 and the histogram of pixel values of the sub region R12." [0035]).
Aoyagi fails to explicitly teach wherein the at least one processor is configured to execute the instructions to cause the display device to display the information based on the frequency distribution.
However, in the same field of endeavor, Douglas teaches wherein the at least one processor is configured to execute the instructions ("Furthermore, it will be understood by those of ordinary skill in the art that the computer-executable instructions may be executed on a variety of tangible processor devices ... The corresponding machines and processes are therefore enabled and within the scope of the disclosure." [0029]) to
cause the display device to display the information based on the frequency distribution ("The interface may include a statistical representation of the tissue types, possibly including but not limited to a histogram bar chart to depict the volume (e.g., number of voxels per unit volume) of the different types of tissue …" [0040]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the image displaying as taught by Aoyagi with the additional representation of volume histogram as taught by Douglas. By providing quantitative analysis, it would be possible to "help the radiologist understand how a feature of interest such as tumor 502 (e.g., the lobulated mass 106, FIG. 1B) is changing in volume" (see Douglas; [0040]).
Claim 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Aoyagi in view of Douglas, as applied in claim 2, and further in view of Karino (US 2021/0153722 A1; published on 05/27/2021).
Regarding claim 3, Aoyagi in view of Douglas teaches all claim limitations, as applied in claim 2, except wherein the at least one processor is configured to execute the instructions to cause the display device to display the information regarding respective degrees of confidence of the plural options based on the frequency distribution.
However, in the same field of endeavor, Karino teaches wherein the at least one processor is configured to execute the instructions ("... the hardware structure of processing units that execute various types of processing, such as ... the display control unit 15 ... and the display style determination unit 42 ..." [0082]) to cause the display device to display the information regarding respective degrees of confidence of the plural options based on the frequency distribution ("In this case, as illustrated in FIG. 7, when a bounding box is used as the specific geometric shape and the detection accuracy of the region of interest is represented by the size of the bounding box, it is preferable that the size of a bounding box 50 used to provide a notification of the region of interest detected in the center region 43 be larger than the size of a bounding box 51 used to provide a notification of the region of interest detected in the peripheral region 44." [0047]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the image displaying as taught by Aoyagi with the information displaying associated with detection accuracy as taught by Karino. "This allows the user to grasp the detection accuracy using the size of bounding boxes" (see Karino; [0047]).
Regarding claim 4, Aoyagi in view of Douglas and Karino teaches all claim limitations, as applied in claim 3, and Karino further teaches wherein the at least one processor is configured to execute the instructions ("... the hardware structure of processing units that execute various types of processing, such as ... the display control unit 15 ... and the display style determination unit 42 ..." [0082]) to cause the display device to display the plural options in association with the respective degrees of confidence ("In this case, as illustrated in FIG. 7, when a bounding box is used as the specific geometric shape and the detection accuracy of the region of interest is represented by the size of the bounding box, it is preferable that the size of a bounding box 50 used to provide a notification of the region of interest detected in the center region 43 be larger than the size of a bounding box 51 used to provide a notification of the region of interest detected in the peripheral region 44." [0047]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the image displaying as taught by Aoyagi with the information displaying associated with detection accuracy as taught by Karino. "This allows the user to grasp the detection accuracy using the size of bounding boxes" (see Karino; [0047]).
Claim 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Aoyagi, as applied in claim 1, and further in view of Karino.
Regarding claim 5, Aoyagi teaches all claim limitations, as applied in claim 1, except wherein the at least one processor is configured to execute the instructions to cause the display device to highlight, on the displayed medical image, an option of the region of interest having a size selected from sizes of the plural options.
However, in the same field of endeavor, Karino teaches wherein the at least one processor is configured to execute the instructions ("... the hardware structure of processing units that execute various types of processing, such as ... the display control unit 15 ... and the display style determination unit 42 ..." [0082]) to cause the display device to highlight, on the displayed medical image, an option of the region of interest ("For example, a bounding box may be colored ..." [0063]) having a size selected from sizes of the plural options ("… the size of the specific geometric shape used to provide a notification of the region of interest detected in the center region 43 is made different from the size of the specific geometric shape used to provide a notification of the region of interest detected in the peripheral region 44." [0047]; see Fig.7 and 8, box 50 - 53 are plural options).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the image displaying as taught by Aoyagi with the information displaying associated with detection accuracy as taught by Karino. "This allows the user to grasp the detection accuracy using the size of bounding boxes" (see Karino; [0047]).
Regarding claim 6, Aoyagi teaches all claim limitations, as applied in claim 1, except wherein the at least one processor is configured to execute the instructions to cause the display device to highlight, on the displayed medical image, an option of the region of interest having a largest size among the plural options.
However, in the same field of endeavor, Karino teaches wherein the at least one processor is configured to execute the instructions ("... the hardware structure of processing units that execute various types of processing, such as ... the display control unit 15 ... and the display style determination unit 42 ..." [0082]) to cause the display device to highlight, on the displayed medical image, an option of the region of interest ("For example, a bounding box may be colored ..." [0063]) having a largest size among the plural options ("… when a bounding box is used as the specific geometric shape and the detection accuracy of the region of interest is represented by the size of the bounding box, it is preferable that the size of a bounding box 50 used to provide a notification of the region of interest detected in the center region 43 be larger than the size of a bounding box 51 used to provide a notification of the region of interest detected in the peripheral region 44." [0047]; see Fig.7 and 8, box 50, 51 are plural options, and the larger box with color is a highlight in visual effect).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ishii et al. (US 2020/0327979 A1; published on 10/15/2020) (hereinafter "Ishii") teaches a medical image and information processing apparatus and method. Various information is added to the displayed image including labels for identifying a group with certainty factors.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAO SHENG whose telephone number is (571)272-8059. The examiner can normally be reached Monday to Friday, 8:30 am to 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M. Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAO SHENG/ Primary Examiner, Art Unit 3797
Prosecution Insights