DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims and Rejections
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claims 1, 3, 7, 13, 21, and 41 have been amended. Claims 8-10 and 20 have been cancelled. Claims 130-132 are newly added.
Claims 1-7, 11-14, 16, 18, 19, 21, 41, and 130-132 are currently pending.
In light of the claim amendments, the Section 112(b) rejection of claims 8-10 have been withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7, 11-14, 16, 18, 19, 21, 41, and 130-132 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite:
inputting brain-scan images to a machine-learning model that comprises:
[a] an encoder having a plurality of layers configured to generate down-sampled feature maps at a plurality of resolutions - (claims 1, 21, and 41);
[b] a decoder configured to generate a segmentation map based on the down-sampled feature maps - (claims 1, 21, and 41);
[c] a classification branch configured to generate a classification score indicative of ARIA based on the down-sampled feature maps - (claims 1, 21, and 41);
[d] outputting a quantification of ARIA in the brain of the patient based at least in part on at least one of the segmentation map and the classification score- (claims 1, 21, and 41).
Independent claims 1, 21, and 41, as drafted and under their broadest reasonable interpretation, recite a mathematical concept and/or mental process. (MPEP 2106.04(a)(2)(I)). More specifically, claim limitations [a], [b], and [c] recite mathematical concepts. With respect to claim limitation [a], mathematical concepts are necessarily and/or inherently used to generate down-sampled feature maps. These mathematical concepts include convolution to compute local weighted sums and pooling (e.g., max pooling or average pooling) to downsample the feature maps by aggregating certain regions. With respect to claim limitation [b], mathematical concepts are necessarily and/or inherently used to generate the segmentation map. These mathematical concepts include upsampling and convolution for refining the upsampled feature maps. With respect to claim limitation [c] and [d], mathematical concepts are necessarily and/or inherently used to generate the classification score. These classification score and quantification represent a probability of the presence or absence of ARIA or a severity of ARIA. That probability is based on calculations made by the classification model. (see, e.g., the Barkhof Grand Total Score (BGTS) in which the score is based on “twelve sub-scores” of twelve bilateral regions. ([0068], [0069]).
Claim limitations [c] and [d] also recite a mental process because the trained machine-learning model, in a generic computer environment, replicates a doctor’s analysis of a medical image by evaluating and providing a judgment/opinion as to ARIA. (see Recentive Analytics, Inc. v. Fox Corp., No. 2023-2437, Federal Circuit, decided on 18 April 2025: “[C]laims that do no more than apply established methods of machine learning to a new data environment” are not patent eligible.”).
Examiner also notes that claims 1, 21, and 41 are conceptually similar to those in Electric Power Group, LLC v. Alstom, S.A., 830 F.3d 1350, 1356 (Fed. Cir. 2016). The claims in Electric Power Group were found to be patent ineligible because, like the claims in this case, they essentially recited collecting information, analyzing that information, and presenting results of that analysis. “[W]e have treated collecting information, including when limited to particular content (which does not change its character as information), as within the realm of abstract ideas…we have treated analyzing information by steps people go through in their minds, or by mathematical algorithms, without more, as essentially mental processes within the abstract-idea category…[and] we have recognized that merely presenting the results of abstract processes of collecting and analyzing information, without more…, is abstract as an ancillary part of such collection and analysis.” Electric Power Group, 830 F.3d 1353-1354.
Once it is established that the claims recite a judicial exception (i.e., an abstract idea), the next question to consider is whether the claims integrate the judicial exception into a practical application. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. (MPEP 2106.04(d)).
Additional elements should be considered to determine if they integrate the judicial exception into a practical application. Here, the additional elements include: (1) the segmentation map including a plurality of pixel-wise class labels corresponding to a plurality of pixels in the segmentation map; and (2) wherein at least one of the plurality of pixel-wise class labels comprises an indication of ARIA in the brain of the patient.
In this case, the judicial exception is not integrated into a practical application. The additional element/step of (1) is a well-understood, routine, conventional activity/element for machine learning models trained to analyze medical images. (MPEP 2106.05(A)). Segmentation maps are intended to differentiate different tissues based on the class/label assigned to each pixel. “However, the main purpose [of segmentation] remains to analyze a group of pixels or voxels and discriminate them based on subjective characteristics.” Akkineni, Sai Darahas, and S. P. K. Karri. “Deep Learning Algorithms for Brain Image Analysis.” Brain and Behavior Computing. CRC Press, 2021. 267-291) (see also FUJIBAYASHI and GAO discussed below).
The additional element of (2) (i.e., class labels indicating ARIA) does no more than generally link the use of a judicial exception to a particular technological environment or field of use. (MPEP 2106.05(h)). More specifically, the recited training model could be applied to various image segmentation scenarios in which tissues are differentiated from one another but, in this case, it is being applied to differentiate and identify ARIA. (Recentive Analytics, Inc. v. Fox Corp., No. 2023-2437, Federal Circuit, decided on 18 April 2025: “[C]laims that do no more than apply established methods of machine learning to a new data environment” are not patent eligible.”).
If the claims recite a judicial exception and do not integrate that exception into a practical application, as is the case here, the next question is whether the claims include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims do not. A shared quality of the additional elements/steps (1) and (2) are that they do not recite any meaningful limitation that transforms the judicial exception into a patent-eligible application. (MPEP 2106.05(II)). As explained above, the additional element/step of (1) is a well-understood, routine, conventional activity. (MPEP 2106.05(A)). The additional element of (2) only generally links the judicial exception to a particular technological environment.
Accordingly, claims 1, 21, and 41 do not include patent-eligible subject matter.
Dependent claims 2-7, 11-14, 16, 18, and 19 also fail to recite patent-eligible subject matter.
For example, claims 2-4 (i.e., ARIA being associated with microhemorrhages, hemosiderin deposits, edema, or sulcal effusion or the patient having Alzheimer’s disease) and claims 7 (i.e., the anti-Aβ antibody being a particular anti-Aβ antibody or the anti-ARIA treatment including anti-ARIA antibodies) and claims 11-12 (i.e., type of brain-scan images or particular type of MR image) recite limitations that amount to no more than generally linking the judicial exception to a field-of-use or technological environment. (MPEP 2106.05(h)).
Claim 5 (i.e., determining a dosage adjustment in response to quantification) and claim 6 (i.e., terminating or temporarily suspending use in response to quantification) are also examples of mental processes (e.g., concepts performed in the human mind, such as observation, evaluation, judgment, opinion). (MPEP 2106.04(a)). Claims 5 and 6 are typical decisions made by the doctor after analyzing and evaluating the MRI images of the brain.
Claims 13, 14, and 16 (i.e., encoder and decoder), claim 18 (i.e., trained using image augmentation), and claim 19 (i.e., pixel-wise class label) recite well-understood, routine, conventional activities/elements for machine learning models that are trained to analyze medical images, as explained below with respect to GAO in the Section 103 rejections. (MPEP 2106.05(A)).
Claims 130-132 (i.e., using down-sampled features maps at different resolutions with local and global information) recite well-understood, routine, conventional activities/elements for machine learning models that are trained to analyze medical images, as explained below with respect to ZHOU in the Section 103 rejections. (MPEP 2106.05(A)).
Accordingly, claims 1-7, 11-14, 16, 18, 19, 21, 41, and 130-132 are rejected for lacking patent-eligible subject matter.
RESPONSE TO APPLICANT’S ARGUMENTS
Applicant’s arguments filed on April 27, 2026 have been fully considered regarding the Section 101 rejection, but they are not persuasive:
Applicant argues that the claims do not recite a mathematical concept if it is only based on or involves a mathematical concept. (p.10 of Response).
However, a mathematical concept need not be expressed in mathematical symbols, because ‘[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula.’ In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). (MPEP 2106.04(a)(2), I; see also Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a “process of organizing information through mathematical correlations” are directed to an abstract idea).
As explained above, claim limitations [a], [b], [c], and [d] are not merely based on or involve mathematical concepts as terms like “down-sampled,” “segmentation,” and “classification score” require operations on data (i.e., brain-scan images, etc.). To be clear, it is necessary for the machine-learning model to use algorithms and perform mathematical calculations in order to generate the down-sampled feature maps, the segmentation map, and the classification score. Indeed, the classification score is a calculated probability or severity of ARIA.
Applicant argues that the claims do not recite a mental process because it contains limitations that cannot be practically performed in the human mind. (p.11 of Response).
However, courts do not distinguish “between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer.” (MPEP 2106.04(a)(2), III). “Claims can recite a mental process even if they are claimed as being performed on a computer.” (Id).
“In evaluating whether a claim that requires a computer recites a mental process, examiners should carefully consider the broadest reasonable interpretation of the claim in light of the specification. For instance, examiners should review the specification to determine if the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.” (MPEP 2106.04(a)(2), III).
As explained above, claim limitations [c] and [d] recite a mental process because the trained machine-learning model, in a generic computer environment, replicates a doctor’s analysis of a medical image by evaluating and providing a judgment/opinion as to ARIA. (see Recentive Analytics, Inc. v. Fox Corp., No. 2023-2437, Federal Circuit, decided on 18 April 2025: “[C]laims that do no more than apply established methods of machine learning to a new data environment” are not patent eligible.”).
Applicant argues that the claims integrate the judicial exception into a practical application by covering a particular technical solution to the technical problems in machine-learning that relate to subtle imaging features and limited training data. (p.12 of Response). Applicant refers to the 2024 Guidance Update on Patent Subject Matter Eligibility which states that an important consideration is to the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome. (p.5 of Response, bolded portion).
However, the claim itself must reflect the disclosed improvement in technology. Intellectual Ventures I LLC v. Symantec Corp.,838 F.3d 1307, 1316, 120 USPQ2d 1353, 1359 (Fed. Cir. 2016) (patent owner argued that the claimed email filtering system improved technology by shrinking the protection gap and mooting the volume problem, but the court disagreed because the claims themselves did not have any limitations that addressed these issues) (p.3 of 2024 Guidance Update).
In this case, it is not clear that the claim reflects the disclosed improvement. Although Applicant cites several passages of the disclosure, (see, e.g., pp. 13-14), many of these passages include bolded language not found in the claim limitations. As such, it is not clear if the claim covers the disclosed improvement in technology. For example, while the claims recite that the “encoder is trained using multi-task learning,” there are no further limitations that clarify the meaning of multi-task learning.
Furthermore, while it is not necessary to recite, in the claim, the particular problems or challenges addressed by the claimed invention, such limitations (i.e., additional elements) support the notion that the judicial exception has been integrated into a practical application. Applicant alleges that claim 1 provides a technical solution to the problem of accurately identifying small, localized ARIA lesions using an ML model where there is limited training data. (p.13 of Response). However, this problem is not reflected in the claim. The claims do not require that the ML model is trained using brain-scan images of individuals suspected of having ARIA or at least trained on data that represents brain-scan images indicating ARIA (i.e., limited training data). Moreover, the method is not limited to analyzing images of patients where there is a concern for ARIA (i.e., images that include features that are challenging to segment, detect, and quantify). Lastly, while the claim limitations include outputting a quantification of ARIA, the term amyloid related imaging abnormalities (ARIA) essentially encompasses swelling or bleeding within the brain. In other words, the broadest reasonable interpretation of claim 1 includes ML models that are trained to identify swelling or hemorrhaging within brain-scan images, regardless of whether the patient is suspected of having ARIA and regardless of whether the swelling or hemorrhaging is amyloid-related.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 11, 12-14, 16, 19-21, and 41 are rejected under 35 U.S.C. 103 as being unpatentable over a translation of Int’l. Publ. No. WO 2023/145953 A1 (hereinafter “FUJIBAYASHI”) in view of U.S. Patent Appl. Publ. No. 2020/0349697 A1 (hereinafter “GAO”).
FUJIBAYASHI teaches a system and a method that are capable of “automatically providing information on an abnormal portion” of a brain using MRI images. (p.1, lines 32-33 and lines 19-20). “[A] computer program causes a computer to acquire an MRI image, specify a signal value of the acquired MRI image, and obtain the specified signal value to display the abnormal part based on and execute the process…. information on an abnormal portion can be automatically provided from an MRI image.” (p.1, lines 37-41). The system and method can be used to evaluate amyloid related imaging abnormalities (ARIA). (p.4, lines 23-31).
With respect to claim 1, FUJIBAYASHI teaches a method for quantifying amyloid related imaging abnormalities (ARIA) in a brain of a patient by one or more computing devices “[A] computer program causes a computer to acquire an MRI image, specify a signal value of the acquired MRI image, and obtain the specified signal value to display the abnormal part based on and execute the process…. information on an abnormal portion can be automatically provided from an MRI image.” (p.1, lines 37-41). With respect to amyloid related imaging abnormalities (ARIA), see p.4, lines 23-31, which is discussed below. With respect to quantifying ARIA, Figures 7-10 illustrate screen displays the present the number and sizes of different edema sites and microhemorrhages. The method includes:
accessing a set of one or more brain-scan images associated with the patient. “The MRI apparatus 10 is an apparatus capable of capturing a tomographic image using a magnetic resonance phenomenon, and can obtain an MRI image (also referred to as an MR image). (p.2, lines 7-8). “The image data server 100 records MRI images for each patient.” (p.2, line 32). The images are part of a set. “The example in the figure indicates that the 128th slice image of 256 slice images is displayed.” (p.6, lines 16-17).
inputting the set of one or more brain-scan images into a model to generate a segmentation map based on the set of one or more brain-scan images. “When an MRI image is input to the brain tissue identifying section 57, the brain tissue identifying section 57 identifies which tissue each pixel of the MRI image belongs to.” (p.4, lines 5-6). “Also, the brain tissue identifying unit 57 may identify brain tissue using a segmentation method. Specifically, using the Bayesian estimation algorithm from the MRI image, a mask image is generated for each tissue….” (p.3, lines 55-57). Notably, the brain tissue identifying section 57 may use a learning model generated by “machine learning…, for example, U-Net, GAN (Generative Adversarial Network), SegNet, etc. may be used.” (p.3, lines 57-59).
FUJIBAYASHI also teaches that the segmentation map includes a plurality of pixel-wise class labels corresponding to a plurality of pixels in the segmentation map. With respect to the brain tissue identifying section 57, FUJIBAYASHI teaches that “a mask image is generated for each tissue with the probability that the tissue exists in each pixel region in the image as a pixel value….” (p.3, lines 56-57); With respect to a signal value specifying unit 56, FUJIBAYASHI teaches that “when the signal value of the MRI image is greater than or equal to the first threshold, it can be determined that the set (region) of pixels having the signal value is edema. Similarly, when the signal value of the MRI image is equal to or less than the second threshold, it can be determined that a set (region) of pixels having that signal value is microhemorrhage.” p.4, lines 53-55. NOTE: Examiner is interpreting “an indication of ARIA in the brain” as being taught by identifying a set (region) of pixels as being edema or a microhemorrhage.
With respect to the limitation, inputting into a machine-learning model, FUJIBAYASHI teaches that the operations performed by the brain tissue identifying unit 57 and the signal value specifying unit 56 could be performed by learning models. More specifically, after discussing the brain tissue identifying unit 57 performing segmentation, FUJIBAYASHI then teaches that a learning model could also be used. “Also, a learning model generated by machine learning other than the Bayesian estimation algorithm may be used. For the learning model, for example, U-Net, GAN (Generative Adversarial Network), SegNet, etc. may be used.” (emphasis added) (p.3, line 58 to page 4, line 2). Notably, Figures 3 and 4 correspond to the operations performed by the brain tissue identifying unit 57 and the signal value specifying unit 56. FUJIBAYASHI reiterates that either could be performed by a learning model. “In the examples of FIGS. 3 and 4 described above, the clustering method is used to classify brain tissue, and the signal value of the MRI image is specified for each brain tissue to determine edema and microhemorrhage. Determination of microhemorrhage is not limited to this. For example, a learning model generated by machine learning or a technique based on statistical analysis (for example, discriminant analysis, which is one of techniques for automatically obtaining a threshold value) may be used.” (p.5, lines 14-16). NOTE: FUJIBAYASHI teaches inputting the images into a learning model. “The learning model 61 is generated to output edema region information, microhemorrhage region information, and normal region (neither edema nor microhemorrhage) region information when an MRI image is input. The MRI images input to the learning model 61 may be T2-weighted images, T2*-weighted images, FLAIR images, SWI images, PADRE images, QSM images, or R2* images.” (p.5, lines 19-23)).
PNG
media_image1.png
200
400
media_image1.png
Greyscale
outputting a quantification of ARIA in the brain of the patient based at least in part on at least one of the segmentation map and the classification score. Figures 9 and 10 are screens that display detection results. Figure 9 is shown here and shows information on edema. “In the detection results shown in FIG. 9, one edema was detected at time t1, four edemas were detected at time t2, and two edemas were detected at time t3.” (p.7, lines 24-25). Figure 10 shows information on microbleeds. “In the detection results shown in FIG. 10, two microbleeds were detected at time t1, eight microbleeds were detected at time t2, and six microbleeds were detected at time t3.” (p.8, lines 18-19).
FUJIBAYASHI teaches that the method could be used to monitor ARIA. “ARIA includes edema with fluid accumulation (ARIA-E) and small hemorrhages on the brain called cerebral microhemorrhages (ARIA-H). Cerebral microhemorrhages are small hemosiderin deposits. ARIA can be specified, for example, on the condition that the magnetic susceptibility is equal to or greater than a predetermined magnetic susceptibility threshold and that the size of the abnormal portion is the specified ARIA size. The specified size of ARIA can be ARIA-H if it is 1 cm or less, and ARIA-E if it is over 1 cm. In particular, if the size of the ARIA is 5 cm or less, the severity of the ARIA can be mild, if it is between 5 cm and 9 cm, it is moderate, and if it is greater than 9 cm, it can be severe.” (p.4, lines 25-31). “With the above configuration, the number of microbleeds can be automatically quantitatively monitored from MRI images, and the increase or decrease in the number of microbleeds at multiple time points can be tracked. As a result, it is possible to automatically quantify microbleeds without being troublesome and unaffected by the experience of doctors, and to understand the progression of lesions and administer appropriate medication under pathological management that monitors changes over time. It can be used as an index for judgment.” (p.8, lines 42-47).
While FUJIBAYASHI teaches inputting the brain-scan images into a machine-learning model (see, e.g., p.4, lines 5-6 and p.3, lines 55-57), FUJIBAYASHI does not explicitly teach the particular limitations related to the machine-learning model. More specifically, FUJIBAYASHI does not teach (1) an encoder comprising a plurality of layers configured to generate a plurality of down-sampled feature maps at a plurality of resolutions, wherein the encoder is trained using multi-task learning, (2) a decoder configured to generate the segmentation map on the plurality of down-sampled feature maps, and (3) a classification branch configured to generate a classification score indicative of ARIA based on the plurality of down-sampled feature maps at the plurality of resolutions. Nonetheless, each of these is a well-known feature of machine-learning models that analyze medical images as explained below with respect to GAO. Moreover, while FUJIBAYASHI teaches identifying brain tissue using a “segmentation method” and that a set (region) of pixels can correspond to edema or a microhemorrhage, (p.3, lines 55-57), it is not clear that FUJIBAYASHI teaches (4) at least one of the plurality of pixel-wise class labels comprising an indication of ARIA. Nonetheless, using pixel-wise class labels is known as explained below with respect to GAO.
In the same field of endeavor, GAO teaches systems and methods that are configured to detect intracerebral hemorrhages (ICH) and “use an end-to-end multi-task learning model for modeling head scan images to solve ICH detection and segmentation problems.” (Abstract and [0022]). “The disclosed systems and methods provide several improvements over conventional approaches. First, the learning model can perform ICH detection and segmentation tasks simultaneously. This enables information sharing and complementation between the two different but closely related tasks. Modules of the learning model are jointly optimized, which can preserve the overall performance of the two tasks while reducing time consumption in both training and prediction stages...Third, the model can be flexible on the type of ICH classification labels it predicts and supports different training scenarios.” ([0023]). While the main embodiment receives images from head computed tomography (CT) scans ([0005]), GAO’s system can also be applied to other “imaging modalities suitable for head scans, including, e.g., Magnetic Resonance Imaging (MRI)….” ([0024]). GAO also notes that “[b]ased on a bleeding location in a brain, ICH can be further categorized into 5 subtypes: epidural hemorrhage (EDH), subdural hemorrhage (SDH), subarachnoid hemorrhage (SAH), cerebral parenchymal hemorrhage (CPH) and intraventricular hemorrhage (IVH). In some embodiments, ICH subtype labels on slice-level or subject-level may be included in training data.”
Figure 2 illustrates GAO’s end-to-end multi-task learning model. GAO teaches an encoder module 202 and a decoder module 204. The encoder module 202 is trained to generate a plurality of down-sampled feature maps based on the set of one or more brain-scan images. “As shown in FIG. 2, encoder module 202 may include a sequence of convolution/pooling layers [i.e., a plurality of layers] to extract task-relevant features from the image slices, e.g., head CT scan slices.” ([0038]) “In FIG. 2, encoder module 202 employs a VGG architectures as an example to illustrate a feature map extraction procedure. For example, convolutional layers use multiple 3×3 kernel-sized filters and pooling layers use 2×2 size filters.” ([0039]). The
PNG
media_image2.png
567
917
media_image2.png
Greyscale
ConvRNN module 206 enhances “the quality of feature maps generated from encoder module 202.” ([0040]).
Accordingly, GAO teaches an encoder comprising a plurality of layers configured to generate a plurality of down-sampled feature maps at a plurality of resolutions, wherein the encoder is trained using multi-task learning.
GAO also teaches a decoder that is configured to generate a segmentation map based on the plurality of down-sampled feature maps. “Decoder module 204 is used to combine feature maps of different granularities to generate segmentation masks.” ([0038]). Note that the decoder module in 204 has multiple rectangular steps, each of which receives feature maps from the encoder module (as indicated by the arrows at each level). Moreover, feature maps from the encoder are used by ConvRNN module. “Consistent with the present disclosure, ConvRNN module 206 may be used to learn contextual information between adjacent image slices across axial axis and enhance the quality of feature maps generated from encoder module 202.” ([0040]). The decoder module also receives the features maps from the ConvRNN module. “Feature maps generated from ConvRNN module 206 are also used by decoder module 204 to produce segmentation masks.” ([0042]).
Accordingly, GAO teaches a decoder configured to generate the segmentation map on the plurality of down-sampled feature maps.
GAO also teaches a classification branch configured to generate a classification score based on the plurality of down-sampled feature maps at the plurality of resolutions. “The classification module then utilizes the output feature maps from the ConvRNN module to generate slice-level and subject-level classification results..” ([0038]). Notably, the output feature maps from ConvRNN module 206 are based on the down-sampled feature maps at the plurality of resolutions from the encoder module. “…ConvRNN module 206 may be used to learn contextual information between adjacent image slices across axial axis and enhance the quality of feature maps generated from encoder module 202.” ([0040]; See also Figure 2 and [0009]: “…detect the ICH [intracerebral hemorrhage] of the subject using the classifier based on the extracted feature maps of the image slices and the contextual information…”).
The classification module can “generate slice-level or subject-level ICH predictions.” ([0038]). This “can be either an ICH identification result or an ICH subtype label depending on the ground truth label types used in the model training.” ([0056]).
Accordingly, GAO teaches a classification branch that is configured to generate a classification result based on the plurality of down-sampled feature maps at the plurality of resolutions.
NOTE: Examiner is interpreting “indication of ARIA” as being taught by the pixels in GAO that are classified as being part of a bleeding volume. ARIA includes edema and microhemorrhages that are attributed to treatment with anti-amyloid-beta (anti-A3) antibodies. ARIA is only determined after monitoring the patient before and during treatment to identify any new microhemorrhages or new/growing sites of edema. (see, e.g., CUMMINGS discussed below). FUJIBAYASHI teaches a system that is capable of monitoring edema and microhemorrhages over time (i.e., capable of detecting ARIA) such that identifying new edema and microhemorrhages means the pixels are indicative of ARIA.
It would have been obvious to one having ordinary skill in the art at the time of filing to modify or replace the learning model of FUJIBAYASHI in order to have an end-to-end multi-task learning model, as taught in GAO, that includes the encoder, decoder, and classification branch and that generates a segmentation map that is based on a set of one or more brain-scan images. One would have been motivated to use the system of GAO because it provides several improvements over conventional approaches, such as performing ICH detection and segmentation tasks simultaneously and being flexible on the type of ICH classification labels it predicts. There would have been a reasonable expectation of success as GAO shows that the system can detect ICH and is flexible enough to also, when trained with MRI images, identify ARIA edema.
With respect to using pixel-wise class labels, GAO teaches that “[t]he training images are previously segmented or annotated by expert operators with each pixel/voxel classified and labeled, e.g., with value 1 if the pixel/voxel indicates a bleeding or value 0 if otherwise. In some embodiments, instead of binary values, the ground truth data may be probability maps where each pixel/voxel is associated with a probability value indicating how likely the pixel/voxel indicate a bleeding.” ([0030]). “The trained learning model may be used by image processing device 103 to detect ICH in new head scan images….” ([0034]). GAO uses the trained model “to perform one or more of: (1) predict whether ICH exists, (2) predict the subtype of ICH, and (3) determine segmentation masks of the image optionally with an estimated bleeding volume.” ([0037]). “[T]he segmentation mask can include but not limited to the following examples: 1) binary ICH masks; 2) detailed ICH subtype masks; 3) ICH or subtype masks together with other desired labels….” ([0036]). “In some embodiments, the decoder module may produce a probability map indicating the probability each pixel in the image slice belongs to a bleeding region. Processor 308 may then perform a thresholding to obtain a segmentation mask. For example, processor 308 may set pixels with probabilities above 0.8 as 1 (i.e., belong to a bleeding region) and the remaining pixels as 0 (i.e., not belong to a bleeding region).” ([0056]).
Accordingly, GAO teaches that the segmentation map includes a plurality of pixel-wise class labels that correspond to a plurality of pixels in the segmentation map, wherein at least one of the plurality of pixel-wise class labels comprises an indication of ARIA in the brain of the patient.
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the FUJIBAYASHI segmentation map to include pixel-wise class labels, as taught in GAO, in which at least one of the labels includes an indication of ARIA. Bleeding is indicative of ARIA and FUJIBAYASHI is concerned with identifying the severity of ARIA. One would have been motivated to use segmentation maps with pixel-wise class labels that indicate ARIA in order to provide a more precise or localized map of where ARIA exists. There would have been a reasonable expectation of success as GAO teaches that pixel-wise class labels can include those that identify bleeding.
However, it is not clear that GAO teaches a classification score other than detecting a hemorrhage or subtype. (see, e.g., [0055] of GAO). Nonetheless, FUJIBAYASHI teaches generating a score that is indicative of the severity of ARIA. “The specified size of ARIA can be ARIA-H if it is 1 cm or less, and ARIA-E if it is over 1 cm. In particular, if the size of the ARIA is 5 cm or less, the severity of the ARIA can be mild, if it is between 5 cm and 9 cm, it is moderate, and if it is greater than 9 cm, it can be severe.” (p.4, lines 25-31 of FUJIBAYASHI).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify or replace the classification result of GAO to include a classification score that indicates whether the ARIA is mild, moderate, or severe as taught in FUJIBAYASHI. One would have been motivated to provide a classification score that indicates the severity of ARIA to better inform the user (e.g., doctor). There would have been a reasonable expectation of success as FUJIBAYSHI teaches that ARIA can be graded or scored and GAO teaches that each pixel can be labeled, thereby enabling grading or scoring of the ARIA.
NOTE: Applicant does not define “classification score” other than that the classification score indicates the “presence of ARIA and/or severity of ARIA.” (see, e.g., [0036] of Applicant’s disclosure). FUJIBAYASHI teaches identifying both the presence of ARIA and severity of ARIA and GAO teaches providing a classification result indicating intracerebral hemorrhaging. As such, the prior art teaches providing a classification score that indicates the presence and/or severity of ARIA.
RESPONSE TO APPLICANT’S ARGUMENTS
Applicant’s arguments filed on April 27, 2026 have been fully considered regarding the Section 103 rejection, but they are not persuasive.
Applicant argues that neither FUJIBAYASHI nor GAO teach that the classification score is “based on the plurality of down-sampled feature maps at the plurality of resolutions.” (pp.16-17 of Response). Applicant states that that classification module of GAO “at most operates on a single feature map output from the ConvRNN module” and references Figure 2 in GAO. (emphasis original) (p.17 of Response).
Examiner respectfully disagrees. GAO’s teachings extend beyond what is illustrated in Figure 2. According to the Abstract of GAO, the classification result is based on multiple feature maps. “The system further includes at least one processor configured to extract feature maps from each image slice using the encoder, capture contextual information between adjacent image slices using the bi-directional ConvRNN, and detect the ICH of the subject using the classifier based on the extracted feature maps of the image slices and the contextual information….” (Abstract). Figure 2 and [0038]-[0039] make it clear that the feature maps have different granularities (i.e., resolutions). “As shown in FIG. 2, encoder module 202 may include a sequence of convolution/pooling layers to extract task-relevant features from the image slices….” ([0038]). Each of these convolution/pooling layers generate feature maps of a predetermined granularity (i.e., resolution). (see, e.g., [0042] describing fusing feature maps of “different granularities.”). Each subsequent convolution/pooling layer uses the feature map from the previous layer. “In FIG. 2, encoder module 202 employs a VGG architectures as an example to illustrate a feature map extraction procedure. For example, convolutional layers use multiple 3×3 kernel-sized filters and pooling layers use 2×2 size filters.” Figure 2 shows that the output of one convolution/pooling layer is passed forward to the next convolution/pooling layer. Accordingly, subsequent feature maps of one convolution/pooling layer are based on the feature maps of earlier layers. (see also [0022] describing that the encoder module generates task-relevant feature maps and that these feature maps have different granularities).
Notably, the classification module in GAO uses more than one feature map. “The classification module then utilizes the output feature maps from the ConvRNN module to generate slice-level and subject-level classification results.” ([0022]). As explained above, each of these feature maps from ConvRNN is based on feature maps of different resolutions from the encoder. NOTE: Even if Applicant were correct and that GAO at most operates on a single feature map output from the ConvRNN module, that single feature map from the ConvRNN module is based on multiple feature maps of different resolutions from the encoder. Accordingly, GAO teaches a classification branch that is configured to generate a classification score that is based on a plurality of down-sampled feature maps at the plurality of resolutions.
With respect to claim 2, FUJIBAYASHI teaches that the ARIA is associated with microhemorrhages and hemosiderin deposits (ARIA-H) in the brain of the patient. ”ARIA includes edema with fluid accumulation (ARIA-E) and small hemorrhages on the brain called cerebral microhemorrhages (ARIA-H)…The specified size of ARIA can be ARIA-H if it is 1 cm or less, and ARIA-E if it is over 1 cm.” (p.4, lines 25-29). “In the detection results shown in FIG. 10, two microbleeds were detected at time t1, eight microbleeds were detected at time t2, and six microbleeds were detected at time t3.” (p.8, lines 18-19) (see Figure 10).
With respect to claim 3, FUJIBAYASHI teaches that the ARIA is associated with parenchymal edema or sulcal effusion (ARIA-E) in the brain of the patient. ”ARIA includes edema with fluid accumulation (ARIA-E) and small hemorrhages on the brain called cerebral microhemorrhages (ARIA-H)…The specified size of ARIA can be ARIA-H if it is 1 cm or less, and ARIA-E if it is over 1 cm.” (p.4, lines 25-29). “In the detection results shown in FIG. 9, one edema was detected at time t1, four edemas were detected at time t2, and two edemas were detected at time t3.” (p.7, lines 24-25) (see Figure 9).
With respect to claim 11, FUJIBAYASHI teaches that wherein the set of one or more brain-scan images comprises one or more magnetic resonance imaging (MRI) images, one or more positron emission tomography (PET) images, one or more single-photon emission computed tomography (SPECT) images, one or more amyloid PET images, or any combination thereof. ”The image data server 100 records MRI images for each patient.” (p.2, line 32). The images are clearly part of a set. “The example in the figure indicates that the 128th slice image of 256 slice images is displayed.” (p.6, lines 16-17).
With respect to claim 12, FUJIBAYASHI teaches that the set of one or more brain-scan images comprises one or more fluid-attenuated inversion recovery (FLAIR) images, one or more T2*-weighted imaging (T2*WI) images, one or more T1-weighted imaging (T1WI) images, or any combination thereof. ”As used herein, MRI images are, for example, T2-weighted images, T2*-weighted images, FLAIR (Fluid-Attenuated Inversion Recovery) images, SWI images, QSM images (quantitative magnetic susceptibility mapping), R2* (R2 star) images, and PADRE (Phase Difference Enhanced Imaging) images.” (p.2, lines 14-16).
With respect to claim 13, FUJIBAYASHI does not explicitly teach the claim limitations. However, in the same field of endeavor, GAO teaches that the decoder is configured to generate a plurality of up-sampled feature maps based on the plurality of down-sampled feature maps and generate the segmentation map based on the plurality of up-sampled feature maps. As discussed above, feature maps from the encoder are used by ConvRNN module. “Consistent with the present disclosure, ConvRNN module 206 may be used to learn contextual information between adjacent image slices across axial axis and enhance the quality of feature maps generated from encoder module 202.” ([0040]). The decoder then receives the features maps from the ConvRNN module. “Feature maps generated from ConvRNN module 206 are also used by decoder module 204 to produce segmentation masks.” NOTE: The feature maps from ConvRNN are based on the down-sampled feature maps from the encoder as explained above. Moreover, “the output from ConvRNN module 206 is first 2× upsampled and then fused with feature maps from encoder module 202 that has the same granularity. The fusion operation can improve segmentation performance.” ([0042]).
It would have been obvious to one having ordinary skill in the art at the time of filing to use the encoder and decoder modules taught in GAO. One would have been motivated to use the encoder and decoder modules of GAO because the learning model provides several improvements over conventional approaches, such as performing ICH detection and segmentation tasks simultaneously and being flexible on the type of ICH classification labels it predicts. There would have been a reasonable expectation of success as GAO shows that the system can detect ICH and is flexible enough to also, when trained with MRI images, identify ARIA edema.
With respect to claim 14 (depending from claim 13), GAO teaches that the encoder comprises a neural network. ”[E]ncoder module 202 may be in any suitable convolutional neutral network (CNN) architecture, including but not limited to the CNN component of commonly used image classification architectures such as VGG, ResNet, and DenseNet.” ([0039]) See also claim 7: “wherein the encoder is a Convolutional Neural Network (CNN).”
With respect to claim 16 (depending from claim 13), GAO teaches that the decoder comprises a neural network. “Consistent with some embodiments, the end-to-end multi-task learning model may be a fully convolutional network (FCN) that include an encoder module, a decoder module….” ([0033]). Although not described separately as a neural network, the decoder operates as a neural network as it takes the feature maps and expands them to generate the segmentation map. Moreover, it is known that an encoder-decoder architecture essentially includes two neural networks (i.e., encoder and decoder) connected to one another. (“Encoder-decoder architecture is a fundamental framework used in various fields, including natural language processing, image recognition, and speech synthesis. At its core, this architecture involves two connected neural networks: an encoder and a decoder.” (https://www.larksuite.com/en_us/topics/ai-glossary/encoder-decoder-architecture).
With respect to claim 19, FUJIBAYASHI teaches wherein the at least one of the plurality of pixel-wise class labels comprises an indication of one or more ARIA lesions. NOTE: Applicant’s disclosure refers to ARIA lesions as “areas of diffuse swelling.” ([0089]). FUJIBAYASHI teaches monitoring the number and size of edema in the brain. (see, e.g., Figure 9). “With the above configuration, the number and size of edema can be automatically monitored quantitatively from MRI images, and the increase and decrease in the number and size of edema can be tracked at multiple time points.” (p.8, lines 4-6). Edema may be color-coded and an alert can be displayed if the edema is excessive. “Although not shown, the previously detected edema and the newly detected edema may be displayed in different display modes (for example, by color coding) so that they can be compared…Also, whether the edema (hyperintense area) is chronic, subacute, or acute may be displayed in a comparable manner. Furthermore, depending on the degree of edema, an alert can be displayed on the image.” (p.7, lines 49-56).
In the FUJIBAYASHI-GAO system, the pixels associated with excessive swelling would be labeled as edema (i.e., an indication of ARIA lesions). (See, e.g., in GAO: “As shown in FIG. 4, when the signal value of the MRI image is greater than or equal to the first threshold, it can be determined that the set (region) of pixels having the signal value is edema.” (p.4, lines 51-52). Accordingly, FUJIBAYASHI-GAO system teaches “wherein the at least one of the plurality of pixel-wise class labels comprises an indication of one or more ARIA lesions.”
With respect to claim 21, FUJIBAYASHI also teaches a system including one or more computing devices (“The information processing device 50 can be configured by a computer...” (p.3, line 5)), comprising: one or more non-transitory computer-readable storage media including instructions (“The storage unit 59 stores a computer program 60…” (p.3, line 9)); and one or more processors coupled to the one or more storage media, the one or more processors configured to execute the instructions (“The control unit 51 can execute processing defined by the computer program 60.” (p.3, lines 17-18)). As discussed above with respect to claim 1, FUJIBAYASHI also teaches that the instructions are executed to:
access a set of one or more brain-scan images associated with the patient. “The MRI apparatus 10 is an apparatus capable of capturing a tomographic image using a magnetic resonance phenomenon, and can obtain an MRI image (also referred to as an MR image). (p.2, lines 7-8). “The image data server 100 records MRI images for each patient.” (p.2, line 32). The images are part of a set. “The example in the figure indicates that the 128th slice image of 256 slice images is displayed.” (p.6, lines 16-17).
input the set of one or more brain-scan images into a model to generate a segmentation map based on the set of one or more brain-scan images. “When an MRI image is input to the brain tissue identifying section 57, the brain tissue identifying section 57 identifies which tissue each pixel of the MRI image belongs to.” (p.4, lines 5-6). “Also, the brain tissue identifying unit 57 may identify brain tissue using a segmentation method. Specifically, using the Bayesian estimation algorithm from the MRI image, a mask image is generated for each tissue….” (p.3, lines 55-57). Notably, the brain tissue identifying section 57 may use a learning model generated by “machine learning…, for example, U-Net, GAN (Generative Adversarial Network), SegNet, etc. may be used.” (p.3, lines 57-59).
FUJIBAYASHI also teaches that the segmentation map includes a plurality of pixel-wise class labels corresponding to a plurality of pixels in the segmentation map. With respect to the brain tissue identifying section 57, FUJIBAYASHI teaches that “a mask image is generated for each tissue with the probability that the tissue exists in each pixel region in the image as a pixel value….” (p.3, lines 56-57); With respect to a signal value specifying unit 56, FUJIBAYASHI teaches that “when the signal value of the MRI image is greater than or equal to the first threshold, it can be determined that the set (region) of pixels having the signal value is edema. Similarly, when the signal value of the MRI image is equal to or less than the second threshold, it can be determined that a set (region) of pixels having that signal value is microhemorrhage.” p.4, lines 53-55. NOTE: Examiner is interpreting “an indication of ARIA in the brain” as being taught by identifying a set (region) of pixels as being edema or a microhemorrhage.
With respect to the limitation, input into a machine-learning model, FUJIBAYASHI teaches that the operations performed by the brain tissue identifying unit 57 and the signal value specifying unit 56 could be performed by learning models. More specifically, after discussing the brain tissue identifying unit 57 performing segmentation, FUJIBAYASHI then teaches that a learning model could also be used. “Also, a learning model generated by machine learning other than the Bayesian estimation algorithm may be used. For the learning model, for example, U-Net, GAN (Generative Adversarial Network), SegNet, etc. may be used.” (emphasis added) (p.3, line 58 to page 4, line 2). Notably, Figures 3 and 4 correspond to the operations performed by the brain tissue identifying unit 57 and the signal value specifying unit 56. FUJIBAYASHI reiterates that either could be performed by a learning model. “In the examples of FIGS. 3 and 4 described above, the clustering method is used to classify brain tissue, and the signal value of the MRI image is specified for each brain tissue to determine edema and microhemorrhage. Determination of microhemorrhage is not limited to this. For example, a learning model generated by machine learning or a technique based on statistical analysis (for example, discriminant analysis, which is one of techniques for automatically obtaining a threshold value) may be used.” (p.5, lines 14-16). NOTE: FUJIBAYASHI teaches inputting the images into a learning model. “The learning model 61 is generated to output edema region information, microhemorrhage region information, and normal region (neither edema nor microhemorrhage) region information when an MRI image is input. The MRI images input to the learning model 61 may be T2-weighted images, T2*-weighted images, FLAIR images, SWI images, PADRE images, QSM images, or R2* images.” (p.5, lines 19-23)).
output a quantification of ARIA in the brain of the patient based at least in part on at least one of the segmentation map and the classification score. Figures 9 and 10 are screens that display detection results. Figure 9 is shown here and shows information on edema. “In the detection results shown in FIG. 9, one edema was detected at time t1, four edemas were detected at time t2, and two edemas were detected at time t3.” (p.7, lines 24-25). Figure 10 shows information on microbleeds. “In the detection results shown in FIG. 10, two microbleeds were detected at time t1, eight microbleeds were detected at time t2, and six microbleeds were detected at time t3.” (p.8, lines 18-19).
FUJIBAYASHI teaches that the method could be used to monitor ARIA. “ARIA includes edema with fluid accumulation (ARIA-E) and small hemorrhages on the brain called cerebral microhemorrhages (ARIA-H). Cerebral microhemorrhages are small hemosiderin deposits. ARIA can be specified, for example, on the condition that the magnetic susceptibility is equal to or greater than a predetermined magnetic susceptibility threshold and that the size of the abnormal portion is the specified ARIA size. The specified size of ARIA can be ARIA-H if it is 1 cm or less, and ARIA-E if it is over 1 cm. In particular, if the size of the ARIA is 5 cm or less, the severity of the ARIA can be mild, if it is between 5 cm and 9 cm, it is moderate, and if it is greater than 9 cm, it can be severe.” (p.4, lines 25-31). “With the above configuration, the number of microbleeds can be automatically quantitatively monitored from MRI images, and the increase or decrease in the number of microbleeds at multiple time points can be tracked. As a result, it is possible to automatically quantify microbleeds without being troublesome and unaffected by the experience of doctors, and to understand the progression of lesions and administer appropriate medication under pathological management that monitors changes over time. It can be used as an index for judgment.” (p.8, lines 42-47).
While FUJIBAYASHI teaches inputting the brain-scan images into a machine-learning model (see, e.g., p.4, lines 5-6 and p.3, lines 55-57), FUJIBAYASHI does not explicitly teach the particular limitations related to the machine-learning model. More specifically, FUJIBAYASHI does not teach (1) an encoder comprising a plurality of layers configured to generate a plurality of down-sampled feature maps at a plurality of resolutions, wherein the encoder is trained using multi-task learning, (2) a decoder configured to generate the segmentation map on the plurality of down-sampled feature maps, and (3) a classification branch configured to generate a classification score indicative of ARIA based on the plurality of down-sampled feature maps at the plurality of resolutions. Nonetheless, each of these is a well-known feature of machine-learning models that analyze medical images as explained below with respect to GAO. Moreover, while FUJIBAYASHI teaches identifying brain tissue using a “segmentation method” and that a set (region) of pixels can correspond to edema or a microhemorrhage, (p.3, lines 55-57), it is not clear that FUJIBAYASHI teaches (4) at least one of the plurality of pixel-wise class labels comprising an indication of ARIA. Nonetheless, using pixel-wise class labels is known as explained below with respect to GAO.
In the same field of endeavor, GAO teaches systems and methods that are configured to detect intracerebral hemorrhages (ICH) and “use an end-to-end multi-task learning model for modeling head scan images to solve ICH detection and segmentation problems.” (Abstract and [0022]). “The disclosed systems and methods provide several improvements over conventional approaches. First, the learning model can perform ICH detection and segmentation tasks simultaneously. This enables information sharing and complementation between the two different but closely related tasks. Modules of the learning model are jointly optimized, which can preserve the overall performance of the two tasks while reducing time consumption in both training and prediction stages...Third, the model can be flexible on the type of ICH classification labels it predicts and supports different training scenarios.” ([0023]). While the main embodiment receives images from head computed tomography (CT) scans ([0005]), GAO’s system can also be applied to other “imaging modalities suitable for head scans, including, e.g., Magnetic Resonance Imaging (MRI)….” ([0024]). GAO also notes that “[b]ased on a bleeding location in a brain, ICH can be further categorized into 5 subtypes: epidural hemorrhage (EDH), subdural hemorrhage (SDH), subarachnoid hemorrhage (SAH), cerebral parenchymal hemorrhage (CPH) and intraventricular hemorrhage (IVH). In some embodiments, ICH subtype labels on slice-level or subject-level may be included in training data.”
Figure 2 illustrates GAO’s end-to-end multi-task learning model. GAO teaches an encoder module 202 and a decoder module 204. The encoder module 202 is trained to generate a plurality of down-sampled feature maps based on the set of one or more brain-scan images. “As shown in FIG. 2, encoder module 202 may include a sequence of convolution/pooling layers [i.e., a plurality of layers] to extract task-relevant features from the image slices, e.g., head CT scan slices.” ([0038]) “In FIG. 2, encoder module 202 employs a VGG architectures as an example to illustrate a feature map extraction procedure. For example, convolutional layers use multiple 3×3 kernel-sized filters and pooling layers use 2×2 size filters.” ([0039]). The ConvRNN module 206 enhances “the quality of feature maps generated from encoder module 202.” ([0040]).
Accordingly, GAO teaches an encoder comprising a plurality of layers configured to generate a plurality of down-sampled feature maps at a plurality of resolutions, wherein the encoder is trained using multi-task learning.
GAO also teaches a decoder that is configured to generate a segmentation map based on the plurality of down-sampled feature maps. “Decoder module 204 is used to combine feature maps of different granularities to generate segmentation masks.” ([0038]). Note that the decoder module in 204 has multiple rectangular steps, each of which receives feature maps from the encoder module (as indicated by the arrows at each level). Moreover, feature maps from the encoder are used by ConvRNN module. “Consistent with the present disclosure, ConvRNN module 206 may be used to learn contextual information between adjacent image slices across axial axis and enhance the quality of feature maps generated from encoder module 202.” ([0040]). The decoder module also receives the features maps from the ConvRNN module. “Feature maps generated from ConvRNN module 206 are also used by decoder module 204 to produce segmentation masks.” ([0042]).
Accordingly, GAO teaches a decoder configured to generate the segmentation map on the plurality of down-sampled feature maps.
GAO also teaches a classification branch configured to generate a classification score based on the plurality of down-sampled feature maps at the plurality of resolutions. “The classification module then utilizes the output feature maps from the ConvRNN module to generate slice-level and subject-level classification results..” ([0038]). Notably, the output feature maps from ConvRNN module 206 are based on the down-sampled feature maps at the plurality of resolutions from the encoder module. “…ConvRNN module 206 may be used to learn contextual information between adjacent image slices across axial axis and enhance the quality of feature maps generated from encoder module 202.” ([0040]; See also Figure 2 and [0009]: “…detect the ICH [intracerebral hemorrhage] of the subject using the classifier based on the extracted feature maps of the image slices and the contextual information…”).
The classification module can “generate slice-level or subject-level ICH predictions.” ([0038]). This “can be either an ICH identification result or an ICH subtype label depending on the ground truth label types used in the model training.” ([0056]).
Accordingly, GAO teaches a classification branch that is configured to generate a classification result based on the plurality of down-sampled feature maps at the plurality of resolutions.
NOTE: Examiner is interpreting “indication of ARIA” as being taught by the pixels in GAO that are classified as being part of a bleeding volume. ARIA includes edema and microhemorrhages that are attributed to treatment with anti-amyloid-beta (anti-A3) antibodies. ARIA is only determined after monitoring the patient before and during treatment to identify any new microhemorrhages or new/growing sites of edema. (see, e.g., CUMMINGS discussed below). FUJIBAYASHI teaches a system that is capable of monitoring edema and microhemorrhages over time (i.e., capable of detecting ARIA) such that identifying new edema and microhemorrhages means the pixels are indicative of ARIA.
It would have been obvious to one having ordinary skill in the art at the time of filing to modify or replace the learning model of FUJIBAYASHI in order to have an end-to-end multi-task learning model, as taught in GAO, that includes the encoder, decoder, and classification branch and that generates a segmentation map that is based on a set of one or more brain-scan images and has a plurality of pixel-wise class labels comprising an indication of ARIA. One would have been motivated to use the system of GAO because it provides several improvements over conventional approaches, such as performing ICH detection and segmentation tasks simultaneously and being flexible on the type of ICH classification labels it predicts. There would have been a reasonable expectation of success as GAO shows that the system can detect ICH and is flexible enough to also, when trained with MRI images, identify ARIA edema.
With respect to using pixel-wise class labels, GAO teaches that “[t]he training images are previously segmented or annotated by expert operators with each pixel/voxel classified and labeled, e.g., with value 1 if the pixel/voxel indicates a bleeding or value 0 if otherwise. In some embodiments, instead of binary values, the ground truth data may be probability maps where each pixel/voxel is associated with a probability value indicating how likely the pixel/voxel indicate a bleeding.” ([0030]). “The trained learning model may be used by image processing device 103 to detect ICH in new head scan images….” ([0034]). GAO uses the trained model “to perform one or more of: (1) predict whether ICH exists, (2) predict the subtype of ICH, and (3) determine segmentation masks of the image optionally with an estimated bleeding volume.” ([0037]). “[T]he segmentation mask can include but not limited to the following examples: 1) binary ICH masks; 2) detailed ICH subtype masks; 3) ICH or subtype masks together with other desired labels….” ([0036]). “In some embodiments, the decoder module may produce a probability map indicating the probability each pixel in the image slice belongs to a bleeding region. Processor 308 may then perform a thresholding to obtain a segmentation mask. For example, processor 308 may set pixels with probabilities above 0.8 as 1 (i.e., belong to a bleeding region) and the remaining pixels as 0 (i.e., not belong to a bleeding region).” ([0056]).
Accordingly, GAO teaches that the segmentation map includes a plurality of pixel-wise class labels that correspond to a plurality of pixels in the segmentation map, wherein at least one of the plurality of pixel-wise class labels comprises an indication of ARIA in the brain of the patient.
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the FUJIBAYASHI segmentation map to include pixel-wise class labels, as taught in GAO, in which at least one of the labels includes an indication of ARIA. Bleeding is indicative of ARIA and FUJIBAYASHI is concerned with identifying the severity of ARIA. One would have been motivated to use segmentation maps with pixel-wise class labels that indicate ARIA in order to provide a more precise or localized map of where ARIA exists. There would have been a reasonable expectation of success as GAO teaches that pixel-wise class labels can include those that identify bleeding.
However, it is not clear that GAO teaches a classification score other than detecting a hemorrhage or subtype. (see, e.g., [0055] of GAO). Nonetheless, FUJIBAYASHI teaches generating a score that is indicative of the severity of ARIA. “The specified size of ARIA can be ARIA-H if it is 1 cm or less, and ARIA-E if it is over 1 cm. In particular, if the size of the ARIA is 5 cm or less, the severity of the ARIA can be mild, if it is between 5 cm and 9 cm, it is moderate, and if it is greater than 9 cm, it can be severe.” (p.4, lines 25-31 of FUJIBAYASHI).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify or replace the classification result of GAO to include a classification score that indicates whether the ARIA is mild, moderate, or severe as taught in FUJIBAYASHI. One would have been motivated to provide a classification score that indicates the severity of ARIA to better inform the user of the system. There would have been a reasonable expectation of success as FUJIBAYSHI teaches that ARIA can be graded or scored and GAO teaches that each pixel can be labeled, thereby enabling grading or scoring of the ARIA.
NOTE: Applicant does not define “classification score” other than that the classification score indicates the “presence of ARIA and/or severity of ARIA.” (see, e.g., [0036] of Applicant’s disclosure). FUJIBAYASHI teaches identifying both the presence of ARIA and severity of ARIA and GAO teaches providing a classification result indicating intracerebral hemorrhaging. As such, the prior art teaches providing a classification score that indicates the presence and/or severity of ARIA.
With respect to claim 41, FUJIBAYASHI teaches the identical access, input, and output steps as described above with respect to claim 1. FUJIBAYASHI also teaches a non-transitory computer-readable medium comprising instructions (“The storage unit 59 stores a computer program 60…” (p.3, line 9)) that, when executed by one or more processors of one or more computing devices, cause the one or more processors to perform the above steps. (“The control unit 51 can execute processing defined by the computer program 60.” (p.3, lines 17-18)).
Claims 4-7 are rejected under 35 U.S.C. 103 as being unpatentable over a translation of Int’l. Publ. No. WO 2023/145953 A1 (hereinafter “FUJIBAYASHI”) and U.S. Patent Appl. Publ. No. 2020/0349697 A1 (hereinafter “GAO”) as applied to claim 1 above, and further in view of Cummings, Jeffrey, et al. “Aducanumab: appropriate use recommendations.” The journal of prevention of Alzheimer's disease 8.4 (2021): 398-410 (hereinafter “CUMMINGS”).
With respect to claim 4, FUJIBAYASHI does not explicitly teach that the patient is an Alzheimer's disease (AD) patient having been treated with an anti-amyloid-beta (anti-A3) antibody. However, FUJIBAYASHI teaches that the method could be used to “understand the progression of lesions and administer appropriate medication under pathological management that monitors changes over time…” (p.8, lines 43-45), and ARIA is one risk for Alzheimers.
CUMMINGS teaches appropriate use recommendations for aducanumab, which “has been approved by the US Food and Drug Administration for treatment of Alzheimer’s disease (AD).” (Abstract). “Aducanumab is an amyloid-targeting monoclonal antibody delivered by monthly intravenous infusions. The pivotal trials included patients with early AD (mild cognitive impairment due to AD and mild AD dementia) who had confirmed brain amyloid using amyloid positron tomography.” (Abstract). However, “[a]ducanumab can substantially increase the incidence of amyloid-related imaging abnormalities (ARIA) with brain effusion or hemorrhage.” (emphasis added) (Abstract). As such, part of treating patients with aducanumab includes monitoring with MRI imaging. “The Expert Panel recommends MRIs prior to initiating therapy, during the titration of the drug, and at any time the patient has symptoms suggestive of ARIA.” (Abstract).
PNG
media_image3.png
198
400
media_image3.png
Greyscale
It would have been obvious to use the FUJIBAYASHI-GAO system to monitor a patient with Alzheimer's disease (AD) patient having been treated with an anti-amyloid-beta (anti-A3) antibody. ARIA is a risk of Alzheimers and the FUJIBAYASHI-GAO system is designed to identify edema and microbleeds and monitor a patient over time. One having ordinary skill in the art would have been motivated to use FUJIBAYASHI-GAO system for its intended purpose while monitoring a patient that is receiving an anti-amyloid-beta (anti-A3) antibody, such as aducanumab.
With respect to claim 5 (depending from claim 4), FUJIBAYASHI does not explicitly teach that in response to outputting the quantification of ARIA in the brain of the patient, determining a dosage adjustment of the anti-A3 antibody. However, FUJIBAYASHI does teach enabling the user to see a “medication history” at different times points, (p.7, lines 20-21), and “to understand the progression of lesions and administer appropriate medication under pathological management….” (p.8, lines 43-45).
Nonetheless, Figure 1 of CUMMINGS illustrates the monitoring schedule that should be followed while administering aducanumab to a patient who has met the enrollment criteria to receive aducanumab. An MRI is performed prior to increasing the titration from T4 to T5. If ARIA is discovered, treatment can be suspended. If ARIA is not discovered (i.e., an output with a quantification that suggests no ARIA), the dosage increases from 3 mg/kg to 6 mg/kg (i.e., a dosage adjustment).
With respect to claim 6 (depending from claim 4), FUJIBAYASHI does not explicitly teach that further comprising: in response to outputting the quantification of ARIA in the brain of the patient, terminating or temporarily suspending use of the anti-A3 antibody in the patient. However, FUJIBAYASHI does teach enabling the user to see a “medication history” at different times points, (p.7, lines 20-21), and “to understand the progression of lesions and administer appropriate medication under pathological management….” (p.8, lines 43-45).
CUMMINGS teaches that “[d]ose interruption or treatment discontinuation is recommended for symptomatic ARIA and for moderate-severe ARIA.” (Abstract). “ARIA led to discontinuation from the trials in 6.2% of patients on aducanumab and 0.6% of patients on placebo.” (p.404, top of left column).
It would have been obvious to one having ordinary skill in the art to terminate or temporarily suspend use of the aducanumab if the output from the FUJIBAYASHI-GAO system suggests that the ARIA is not being managed. Edema and/or microhemorrhages can risk the life and well-being of the patient. If the output from the FUJIBAYASHI-GAO system provides evidence that edema and/or microhemorrhages are increasing while on aducanumab, the obvious response would be to terminate or temporarily suspend treatment.
With respect to claim 7 (depending from claim 4), FUJIBAYASHI does not explicitly teach that wherein the anti-A3 antibody is selected from the group consisting of bapineuzumab, solanezumab, aducanumab, gantenerumab, crenezumab, donanemab, and lecanemab.
CUMMINGS teaches appropriate use recommendations for aducanumab, which “has been approved by the US Food and Drug Administration for treatment of Alzheimer’s disease (AD).” (Abstract). “Aducanumab is an amyloid-targeting monoclonal antibody delivered by monthly intravenous infusions.” (Abstract). It would have been obvious to one having ordinary skill in the art to use aducanumab while monitoring a patient as it has been approved for treatment of Alzheimer’s disease.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over a translation of Int’l. Publ. No. WO 2023/145953 A1 (hereinafter “FUJIBAYASHI”) and U.S. Patent Appl. Publ. No. 2020/0349697 A1 (hereinafter “GAO”) as applied to claim 1 above, and further in view of U.S. Patent Appl. Publ. No. 2024/0420333 A1 (hereinafter “DADAR”).
With respect to claim 18, FUJIBAYASHI does not explicitly teach that wherein the one or more machine-learning models is trained using image augmentations.
In the same field of endeavor, DADAR is directed to a classifier that is trained to recognize microbleed voxels within a brain image. (Abstract). While training a learning model for segmentation tasks, DADAR teaches: “Applicant further augmented the microbleed patch dataset by randomly rotating the patches to generate additional training data. The random rotations may be performed on the full slice (not the patches) centering around the microbleed voxel; therefore, the corner voxels in the patches include information from different areas not present in other patches. Matching numbers of novel background patches may also be added to balance the training dataset. The performance of the model may be assessed using the training dataset with no augmentation, and with adding 4, 9, 14, 19, 24, and 29 random rotations to the training set, respectively.” (emphasis added) ([0077]). The data augmentation proved effective. “FIG. 10 shows the average performance of the model with these parameters trained with no augmentation, as well as the same model trained on original data plus data augmented with 4, 9, 14, 19, 24, and 29 random rotations (no augmentation was performed on validation and test sets). All models with data augmentation performed better than the model without any data augmentation.” (emphasis added) (0112]).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the FUJIBABYASHI-GAO learning model by training the learning model with data augmentation, as taught in DADAR. One of ordinary skill in the art would have used data augmentation because, as taught in DADAR, models with data augmentation perform better than models without data augmentation. There would have been a reasonable expectation of success as DADAR teaches that data augmentation can be used with learning models configured for segmentation tasks.
Claims 130-132 are rejected under 35 U.S.C. 103 as being unpatentable over a translation of Int’l. Publ. No. WO 2023/145953 A1 (hereinafter “FUJIBAYASHI”) and U.S. Patent Appl. Publ. No. 2020/0349697 A1 (hereinafter “GAO”) as applied to claim 1 above, and further in view of Zhou, Yue, et al. “Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images.” Medical image analysis 70 (2021): 101918. (Year: 2021) (hereinafter “ZHOU”).
With respect to claim 130, neither FUJIBAYASHI nor GAO teach wherein the classification branch is configured to receive the plurality of down-sampled feature maps at the plurality of resolutions from a plurality of layers of the encoder.
ZHOU teaches a multi-task learning framework that is designed to combine the tasks of tumor segmentation and classification. (Abstract). ZHOU notes that tumors “are challenging due to the significant shape variation of breast tumors and the fuzzy nature of ultrasound images (e.g., low contrast and signal to noise ratio).” (Abstract). “Considering the correlation between tumor classification and segmentation, [ZHOU argues] that learning these two tasks jointly is able to improve the outcomes of both tasks.” ZHOU suggests that “training two tasks jointly in one network to encourage feature sharing between breast tumor classification and segmentation is a promising direction to explore.” (p.2, top left column). “The proposed framework consists of two sub-networks: an encoder-decoder network for segmentation and a light-weight multi-scale network for classification. To account for the fuzzy boundaries of tumors in ABUS images, our framework uses an iterative training strategy to refine feature maps with the help of probability
PNG
media_image4.png
1063
1174
media_image4.png
Greyscale
PNG
media_image5.png
2476
2770
media_image5.png
Greyscale
maps obtained from previous iterations.” (Abstract). Again, ZHOU teaches that “[t]he segmentation and classification tasks share features extracted from the encoding path.” (p.2, left column).
Figures 2(a) and (b) are shown here. ZHOU’s method is as follows: “A classification branch is added to the bottom of the VNet, as illustrated in Fig. 2b. Firstly, feature maps from Stage 4, Stage 5, and Stage 6 are fed into the classification network. Then, we fuse these shared feature maps for the classification task. Finally, we input fused features to the classification branch which has two fully connected (FC) layers and one softmax layer to predict the input volume as benign or malignant.” To be clear, Stages 4 and 5 are part of the encoder and each includes a feature map that is used as an input for the classification path. However, the feature maps of Stages 4 and 5 have a different resolutions. Notably, ZHOU teaches that “low-level features mainly capture shape and boundary information, [whereas] high-level features summarize attributes of different targets and are commonly used in classification tasks.” (p.3, 1st paragraph in Section 3.2).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the classification branch of GAO, as applied to ARIA detection as taught in FUJIBAYASHI, so that the classification branch receives the plurality of down-sampled feature maps at the plurality of resolutions from a plurality of layers of the encoder. One of ordinary skill in the art would have been motivated to modify the system in this manner to improve the classification of ARIA by using the spatial/boundary information and high-level semantic information as taught by ZHOU. There would have been a reasonable expectation of success as GAO and ZHOU both teach joint segmentation and classification task in which the classification branch uses down-sampled feature maps.
With respect to claim 131, neither FUJIBAYASHI nor GAO teach wherein the classification branch is configured to aggregate the plurality of down-sampled feature maps prior to generating the classification score.
However, ZHOU teaches aggregating (i.e., ) the plurality of down-sampled feature maps prior to generating the classification score. This is essentially shown Figure 2(b). ZHOU teaches that one “can observe that low-level features mainly capture shape and boundary information, high-level features summarize attributes of different targets and are commonly used in classification tasks. However, capturing features for small objects can be challenging when network depth increases with more convolution and downsampling operations (Liu et al., 2015). To solve this problem, we design a multi-scale feature concatenation model for the classification task, as shown in Fig. 2b. We connect and fuse feature maps from Stage 4 to Stage 6 in VNet as classification features.” (p.3, 1st paragraph in Section 3.2; see also p.7, Discussion: “We employ a multi-scale feature concatenation network, realized via GAP layers and feature channel concatenations, to fuse features from different VNet stages for classification.”).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the classification branch of GAO, as applied to ARIA detection as taught in FUJIBAYASHI, so that the classification branch connects and fuses (i.e., aggregates) the plurality of down-sampled feature maps at the plurality of resolutions from a plurality of layers of the encoder. One of ordinary skill in the art would have been motivated to modify the system in this manner to improve the classification of ARIA by using the spatial/boundary information and high-level semantic information as taught by ZHOU. There would have been a reasonable expectation of success as GAO and ZHOU both teach joint segmentation and classification task in which the classification branch uses down-sampled feature maps.
With respect to claim 132, neither FUJIBAYASHI nor GAO teach wherein the plurality of down-sampled feature maps at the plurality of resolutions comprise information corresponding to local regions of the set of one or more brain-scan images and information corresponding to global regions of the set of one or more brain-scan images.
ZHOU teaches a multi-task learning framework that is designed to combine the tasks of tumor segmentation and classification. (Abstract). ZHOU teaches that tumors “are challenging due to the significant shape variation of breast tumors and the fuzzy nature of ultrasound images (e.g., low contrast and signal to noise ratio).” (Abstract). Figure 1 illustrates this and note that “[t]umors vary significantly in size and shape with irregular and ambiguous boundaries.” ZHOU suggests that “training two tasks jointly in one network to encourage feature sharing between breast tumor classification and segmentation is a promising direction to explore.” (p.2, top left column). “The proposed framework consists of two sub-networks: an encoder-decoder network for segmentation and a light-weight multi-scale network for classification. To account for the fuzzy boundaries of tumors in ABUS images, our framework uses an iterative training strategy to refine feature maps with the help of probability maps obtained from previous iterations.” (Abstract).
The feature maps at different resolution enable using local information (i.e., shape and boundaries) and global information (i.e., “volume-level classification probability”). “We can observe that low-level features mainly capture shape and boundary information, high-level features summarize attributes of different targets and are commonly used in classification tasks.” (p.3, right column, Section 3.2). Notably, “the input to the multi-task learning network is a 3D ABUS volume and the output is a 3D segmentation probability map with a classification score. The predicted probability maps contain contextual information and can be used to guide the network to focus on tumor regions.” (p.4, left column, Section 3.3).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the encoder of GAO, as applied to ARIA detection as taught in FUJIBAYASHI, so that the plurality of down-sampled feature maps at the plurality of resolutions comprise information corresponding to local regions of the set of one or more brain-scan images and information corresponding to global regions of the set of one or more brain-scan images. One of ordinary skill in the art would have been motivated to modify the system in this manner to improve the classification of ARIA by using the spatial/boundary information and high-level semantic information as taught by ZHOU. There would have been a reasonable expectation of success as GAO and ZHOU both teach joint segmentation and classification task in which the classification branch uses down-sampled feature maps.
Prior Art Made of Record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Siddique, Nahian, et al. “U-net and its variants for medical image segmentation: A review of theory and applications.” IEEE access 9 (2021): 82031-82057 (hereinafter “SIDDIQUE”). SIDDIQUE teaches various encoder-decoder architectures that are used for medical image segmentation and that include down-sampling and up-sampling feature maps.
Chen, Zhiwei, et al. "Multi-scale features for weakly supervised lesion detection of cerebral hemorrhage with collaborative learning." Proceedings of the 1st ACM International Conference on Multimedia in Asia. 2019. (hereinafter “CHEN”). CHEN teaches a “Multi-scale Feature with Collaborative Learning (MFCL) strategy” for identifying small lesions caused by cerebral hemorrhages. (Abstract). CHEN’s system “not only adapts to the characteristics of detecting small lesions but also introduces the global constraint classification objective in training. Specifically, a multi-scale feature branch network and a collaborative learning are designed to locate the lesion area.” (Abstract).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON P GROSS whose telephone number is (571)272-1386. The examiner can normally be reached Monday-Friday 9:00-5:00CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M. Kozak can be reached at (571) 270-5284. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASON P GROSS/ Examiner, Art Unit 3797
/ANNE M KOZAK/ Supervisory Patent Examiner, Art Unit 3797
Prosecution Insights