DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-20 are currently pending and are herein under examination.
Claims 1-20 are rejected.
Claims 2, 10-11 and 17 are objected.
Priority
The instant application does not claim benefit of priority to any earlier filed applications. As such, the effective filing date for claims 1-20 is 28 March 2023.
Information Disclosure Statement
The IDS filed 12/29/2023 follows the provisions of 37 CFR 1.97 and has been considered in full. A signed copy of the list of references cited from this IDS is included with this Office Action.
Drawings
The drawings filed 3/28/2023 are accepted.
Abstract
The abstract of the disclosure is objected to because is recites implied language of “embodiments of the present disclosure”. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
Claim Objections
Claims 2, 10-11 and 17 are objected to because of the following informalities:
Claims 2, 11 and 17 recite “Bayseian” which should be “Bayesian”.
Claim 10, pg. 52, line 5 should recite “by:”.
Claim 10, pg. 53, line 3, should recite “predictions; and”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
35 USC 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims dependent upon a rejected claim are also rejected, unless otherwise noted.
Claim 10 (pg. 53, last line) and claim 13 recite “the histopathological image” which renders the claims indefinite. It is unclear if the recitation refers to “a histopathological image” in claim 10, pg. 52, line 2, or refers to a specific image from the “histopathology images” in claim 10, pg. 52, line 7. Clarify which image is being referenced.
Claim 14 recites “the uncertainty” which renders the claim indefinite. It is unclear if the recitation refers to “an uncertainty” in claim 10, pg. 52, line 3, or refers to “an uncertainty” in claim 10, pg. 53, line 1. Clarify which uncertainty is being referenced.
Claim Rejections - 35 USC § 101
Non-Statutory Subject Matter
Claims 16-20 are rejected under 35 U.S.C. 101 because they are directed to non-statutory subject matter (Step 1: NO). Claims 16-20 recite a computer program product comprising a computer readable storage medium having instructions stored therein. The broadest reasonable interpretation of storing instructions in memory includes transitory forms of signal transmission or “signals per se” when the memory is not recited as “non-transitory”. Signals per se do not fall within a category of statutory subject matter (MPEP 2106.03.I).
Claims 16-20 can be amended to recite statutory subject matter by specifying that the medium is non-volatile, as discussed in specification para. [93-94]. However, this amendment would still result in a rejection of claims 16-20 under 35 U.S.C. 101 for recitation of a judicial exception without significantly more. In the interest of compact prosecution, claims 16-20 are analyzed below under 35 U.S.C. 101 using the Alice/Mayo test as if they recited statutory subject matter.
Statutory Subject Matter
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
Step 1 asks whether the claims recite statutory subject matter. In the instant application, claims 1-9 recite a method, claims 10-15 recite a method, and claims 16-20 recite a CRM. As such, these claims recite statutory subject matter (Step 1: YES).
Step 2A, Prong 1:
Claims that recite statutory subject matter are analyzed under Step 2A, Prong 1 to determine if they recite any concepts that equate to an abstract idea, law of nature or natural phenomena. The instant claims recite the following limitations that equate to one or more categories of judicial exception:
Claims 1 and 16 recite “sampling a plurality of deep neural network models trained using a plurality of histopathological images; determining an uncertainty in predictions based on the plurality of sampled deep neural network models; computing an uncertainty threshold based on the uncertainty in the predictions; categorizing an uncertainty of a histopathological image prediction by comparing an uncertainty associated with the histopathological image prediction with the uncertainty threshold.”
Claims 2, 11 and 17 recite “wherein the plurality of deep neural network models are Bayseian neural network models.”
Claims 3, 12 and 18 recite “wherein the plurality of deep neural network models include dropout-enabled hidden layers.”
Claims 4, 13 and recite “wherein the histopathological image associated with the histopathological image prediction is a whole slide-level image.”
Claims 5, 15 and 19 recite “wherein the uncertainty in predictions is based on a standard deviation associated with output from each of the plurality of sampled deep neural network models.”
Claims 6, 14 and 20 recite “wherein the uncertainty threshold is based on maximizing a Youden's index metric associated with a sensitivity and a specificity.”
Claim 7 recites “categorizing the uncertainty of the histopathological image prediction as high-confidence when an uncertainty associated with the histopathological image prediction is less than the uncertainty threshold.”
Claim 8 recites “determining the histopathological image prediction.”
Claim 9 recites “wherein the plurality of histopathological images and the histopathological image associated with the histopathological image prediction have different domains.”
Claim 10 recites “obtaining therefrom a histopathological image prediction and an uncertainty; comparing the uncertainty to an uncertainty threshold, the uncertainty threshold having been determined by sampling a plurality of deep neural network models trained using histopathological images, determining an uncertainty in predictions based on the plurality of sampled deep neural network models, and computing the uncertainty threshold based on the uncertainty in the predictions.”
Claim 16 recites “sampling a plurality of deep neural network models trained using a plurality of histopathological images; determining an uncertainty in predictions based on the plurality of sampled deep neural network models; computing an uncertainty threshold based on the uncertainty in the predictions; categorizing an uncertainty of a histopathological image prediction by comparing an uncertainty associated with the histopathological image prediction with the uncertainty threshold.”
Limitations reciting a mental process.
Claims 1, 5-8, 10, 14-16 and 19-20 contain limitations recited at such a high level of generality that they equate to a mental process because they are similar to the concepts of collecting information, analyzing it, and displaying certain results of the collection and analysis in Electric Power Group, LLC, v. Alstom (830 F.3d 1350, 119 USPQ2d 1739 (Fed. Cir. 2016)), which the courts have identified as concepts that can be practically performed in the human mind. The paragraphs below discuss the broadest reasonable interpretation (BRI) of the limitations in these claims that recite a mental process.
Regarding claims 1, 10 and 16, sampling models includes making a selection based on desired criteria, or calculating a posterior distribution of model, as recited in specification para. [59]. Determining an uncertainty in predictions includes performing on pen and paper calculations using equations 1 and 6 recited in specification para [60] and [64] based on previously derived predictions from the sampled DNNs. Computing an uncertainty threshold includes performing on pen and paper calculations using equations 3, 7, or 9 recited in specification para. [62] and [65-66] based on the calculated uncertainty in predictions. Categorizing an uncertainty of an image prediction includes comparing numbers to a numerical threshold which requires mental determinations. Alternatively, it includes performing on pen and paper calculations using equations 4 or 8 recited in specification para. [63] and [66].
Claim 10 limitation of obtaining a prediction and uncertainty from the DNN model includes collecting data previously generated from the DNN. This is because the DNN is not required to perform any algorithmic steps.
Claims 5, 15 and 19 include calculating on pen and paper a standard deviation of the predictions from the sampled DNNs. Claims 6, 14 and 20 include performing on pen and paper the calculations of equations 2-3 as recited in specification para. [61-62]. Claim 7 includes calculating on pen and paper equations 4 and 8 as recited in specification para. [63] and [66] or a mental determination based on comparing numerical values to a threshold. Claim 8 includes a mental process of analyzing an image and making a prediction regarding its pathology.
Limitations reciting a mathematical concept.
Claims 1, 5-7, 10, 14-16 and 19-20 recite limitations that equate to a mathematical concept because they are similar to the concepts of organizing and manipulating information through mathematical correlations in Digitech Image Techs., LLC v Electronics for Imaging, Inc. (758 F.3d 1344, 111 U.S.P.Q.2d 1717 (Fed. Cir. 2014)), which the courts have identified as mathematical concepts. The paragraphs below discuss the broadest reasonable interpretation (BRI) of the limitations in these claims that recite a mathematical concept.
Regarding claims 1, 10 and 16, sampling models includes using a Monte Carlo dropout or Gaussian mixture, as described in specification para. [59]. Determining uncertainty in predictions includes performing calculations using equations 1 and 6 as recited in specification para. [68]. Computing an uncertainty threshold includes performing calculations using equations 3, 7, or 9 as recited in specification para. [68]. Categorizing a histopathology slide includes performing calculations using equations 4 or 8 as recited in specification para. [68].
Claims 5, 15 and 19 include calculating a standard deviation of the predictions from the sampled DNNs. Claims 6, 14 and 20 include performing calculations using equations 2 and 3 as recited in specification para. [61-62]. Claim 7 includes performing calculations using equations 4 and 8 as recited in specification para. [63] and [66].
Limitations included in the recited judicial exception.
Claims 2-3, 11-12 and 17-18 are included in the judicial exception in claims 1, 10 and 16 of sampling DNNs and determining an uncertainty in predictions because they further limit the type of DNNs. The DNNs are not required to perform any action, unlike in claim 8. Rather, the BRI of sampling and determining includes collecting data previously generated using DNNs.
Claims 4, 9 and 13 are included in the judicial exception in claims 1 and 10 of sampling the DNNs and categorizing an uncertainty because they further limit the plurality of images and the image associated with the image prediction but do not alter the fact that sampling and categorizing recite a judicial exception.
As such, claims 1-20 recite an abstract idea (Step 2A, Prong 1: YES).
Additional Elements:
Once limitations have been identified that recite a judicial exception, the claims are evaluated for additional elements. The additional elements are then analyzed under Step 2A, Prong 2 then Step 2B. The instant claims recite the following additional elements:
Claim 8 recites “from a deep neural network model”
Claim 10 recites “providing a histopathological image to a deep neural network model and outputting a pathology in the histopathology image based on the comparison.”
Claim 11 recites “the deep neural network model are Bayesian neural networks.”
Claim 12 recites “the deep neural network model each include dropout-enabled hidden layers.”
Claim 16 recites “A computer program product for assessing uncertainty of a histopathological image prediction comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:”
Claims 17-20 recite “The computer program product of claim 16”
These above recited additional elements are analyzed below under both Step 2A, Prong 2 and Step 2B:
Step 2A, Prong 2:
Claims found to recite a judicial exception under Step 2A, Prong 1 are then further analyzed to determine if the claims as a whole integrate the recited judicial exception into a practical application or not (Step 2A, Prong 2). The judicial exception is not integrated into a practical application because the claims do not recite additional elements that reflect an improvement to a computer, technology, or technical field (MPEP § 2106.04(d)(1) and 2106.5(a)), require a particular treatment or prophylaxis for a disease or medical condition (MPEP § 2106.04(d)(2)), implement the recited judicial exception with a particular machine that is integral to the claim (MPEP § 2106.05(b)), effect a transformation or reduction of a particular article to a different state or thing (MPEP § 2106.05(c)), nor provide some other meaningful limitation (MPEP § 2106.05(e)). Rather, the claims include limitations that equate to an equivalent of the words “apply it” and/or to instructions to implement an abstract idea on a computer (MPEP § 2106.05(f)), insignificant extra-solution activity (MPEP § 2106.05(g)), and field of use limitations (MPEP § 2106.05(h)). The paragraphs below discuss the additional elements recited above in the instant claims.
Claims 16-20 recite a computer readable medium. There are no limitations that CRM requires anything other than a generic computer and/or generic computing system. Therefore, these limitations equate to mere instructions to implement an abstract idea on a generic computer, which the courts have established does not render an abstract idea eligible in Alice Corp. 573 U.S. at 223, 110 USPQ2d at 1983.
Claim 10 limitations of providing an image to a DNN and outputting a pathology in the image equate to insignificant extra-solution activity of necessary data gathering/outputting (MPEP 2106.05(g)(3)). These limitations gather data necessary to perform the judicial exception in claim 10 of comparing the uncertainty and output the result of the judicial exception in claim 10 of the comparison step. Claims 11-12 also recite data gathering because they further limit the DNN in claim 10, and equate to field of use limitations because they confine the DNN to specific types of DNNs.
Claim 8 recites “from a deep neural network model”, the BRI of this limitation include it being mere instructions to implement an abstract idea on a generic computer. MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. The two following paragraphs provide analysis under these considerations.
The deep neural network (DNN) performs the abstract idea of “determining the histopathological image prediction”. The DNN is used to generally apply the abstract idea without placing any limits on how the DNN functions. Rather, this limitation only recites the outcome of “determining the histopathological image prediction”. This limitation does not include any details about how the abstract idea is accomplished. See MPEP 2106.05(f).
This limitation also indicates a field of use or technological environment in which the judicial exception is performed. Although “from a deep neural network model” limits the abstract idea, it merely confines the use of the abstract idea to a particular technological environment (i.e., DNNs) and thus fail to add an inventive concept to the claims. See MPEP 2106.05(h).
As such, claims 1-20 are directed to an abstract idea (Step 2A, Prong 2: NO).
Step 2B:
Claims found to be directed to a judicial exception are then further evaluated to determine if the claims recite an inventive concept that provides significantly more than the judicial exception itself (Step 2B). These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because these claims recite additional elements that equate to instructions to apply the recited exception in a generic way and/or in a generic computing environment (MPEP § 2106.05(f)) and to well-understood, routine and conventional (WURC) limitations (MPEP § 2106.05(d)). The paragraphs below discuss the additional elements recited above in the instant claims.
Claims 16-20 recite a computer readable medium. There are no limitations that the CRM requires anything other than a generic computer and/or generic computing system. Therefore, these limitations equate to instructions to implement an abstract idea on a generic computing environment, which the courts have established does not provide an inventive concept in Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015).
Claims 16-20 recite storing instructions in a CRM which equates to storing information in memory, which the courts have established as a WURC function of a generic computer in Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
Claim 10 recites providing an image to a DNN and outputting a pathology in the image. Claim 10 does not require performing any algorithmic steps using the DNN. Rather, the BRI of these limitations includes transmitting data to a DNN and transmitting an output of an image. Thus, these limitations equate to receiving/transmitting data over a network, which the courts have established as WURC limitation of a generic computer in buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014). Claims 11-12 also equate to transmitting data over a network because they limit the type of DNN data is transmitted to but does not negate the fact that data is being transmitted.
Claim 8 limitation of “from a deep neural network model” equates to instructions to “apply” the abstract idea, which cannot provide an inventive concept (MPEP 2106.05(f)). See above in section Step 2A, Prong 2 for further discussion.
When the additional elements of claims 10-12 are viewed in combination, they equate to WURC limitations of DNNs in combination with generic computer components/functions as taught by Jospin et al. (“Jospin”; IEEE Computational Intelligence Magazine 17, no. 2 (2022): 29-48) and Pearce et al. (“Pearce”; arXiv preprint arXiv:1811.12188 (2018)). Jospin reviews literature and toolset design to implement, train, use and evaluate Bayesian neural networks (BNN) based on DNN architecture (abstract) (pg. 3, col. 1, last para.). BNNs are considered a special case of ensemble learning (pg. 3, col. 1, para. 2). Figure 4 shows the architecture of BNNs which includes Monte Carlo dropout-enabled hidden layers (sec. E.1). Pearce discloses BNN ensembles that use MC dropout on activation functions (title) (Figure 1) (sec. 4).
When these additional elements are considered individually and in combination, they do not provide an inventive concept because they equate to mere instructions to apply the judicial exception in a generic computer, WURC functions/components of a generic computer, and are WURC limitations of DNNs and BNNs as taught above by Jospin and Pearce. Therefore, these additional elements do not transform the claimed judicial exception into a patent-eligible application of the judicial exception and do not amount to significantly more than the judicial exception itself (Step 2B: No).
As such, claims 1-20 are not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dolezal et al.
(“Dolezal”; Nature communications 13, no. 1 (2022): 6572).
Dolezal constitutes prior art under 35 U.S.C. 102(a)(1) even though it is within the one-year grace period because it names authors that are not joint inventors.
The bold and italicized text below are the limitations of the instant claims, and the italicized text serves to map the prior art onto the instant claims.
Claims 1 and 16:
A method for assessing uncertainty of a histopathological image prediction, the method comprising:
A computer program product for assessing uncertainty of a histopathological image prediction comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
Dolezal discloses uncertainty-informed deep learning models enable high-confidence predictions for digital histopathology (title). Various computer-implemented software were utilized indicating use of computers that inherently comprises processors and memory (pg. 12, col. 1, sec. Code Availability).
sampling a plurality of deep neural network models trained using a plurality of histopathological images;
Bayesian neural networks (BNN) were used (pg. 9, col. 2, para. 1). Dolezal recites “BNN is a specific version of the ensemble method which differs from alternatives such as Deep Ensembles in the way that members of the ensemble are sampled: sampling is performed from a posterior distribution of models conditioned on the training data” (pg. 10, col. 2, para. 1).
determining an uncertainty in predictions based on the plurality of sampled deep neural network models;
Equations 1, 5 and 6 determine uncertainty for the sampled BNNs.
computing an uncertainty threshold based on the uncertainty in the predictions;
Equations 3, 7 and 9 determine uncertainty thresholds from BNNs uncertainty in predictions.
categorizing an uncertainty of a histopathological image prediction by comparing an uncertainty associated with the histopathological image prediction with the uncertainty threshold.
Equations 4 and 8 determine whether a pathology image has high- or low-confidence based on uncertainty thresholds.
Claim 10:
The claim mapping of claim 1 applies to claim 10. The difference between claims 1 and 10 is that claim 10 requires providing an image to a DNN to obtain an image prediction/uncertainty then outputting a pathology in an image based on a comparison. Dolezal provides images to a DNN and obtains an image prediction (Figure 1). A slide is outputted with a pathology prediction (Figure 6c).
Claims 2, 11 and 17:
BNN ensemble was used (pg. 10, col. 1, last para.).
Claims 3, 12 and 18:
Figure 1 shows BNN with dropout-enabled hidden layers (pg. 10, col. 1, para. 3).
Claims 4 and 13:
Image tiles were extracted from whole-slide images (pg. 10, col. 1, para. 2).
Claims 5, 14 and 19:
Standard deviation was used to represent tile-level uncertainty (Figure 1 caption). Dolezal recites “[t]his is an ensemble method where uncertainty is quantified as the disagreement of the predictions made by different models sampled from an ensemble of neural networks. The disagreement is computed as the standard deviation of the predictions by the sampled neural networks” (pg. 10, col. 1, last para.).
Claims 6, 15 and 20:
The optimal tile- and slide-level uncertainty thresholds were defined as the threshold that maximizes Youden’s index. See equations 3 and 7.
Claim 7:
Dolezal recites “[t]iles from the dataset are separated into high- and low-confidence by whether the tile-level uncertainty falls below or above θtile, respectively” (Figure 1 caption).
Claim 8:
Figures 1 and 4 show histopathology image predictions from a DNN.
Claim 9:
Figure 4 shows uncertainty thresholding improves predictions on external datasets in the setting of domain shift. Models trained on TCGA were validated on lung adenocarcinomas and squamous cell carcinomas from the Clinical Proteomic Tumor Analysis Consortium (Figure 4 caption).
Rejection over Thiagarajan et al.
Claims 1-2, 4-5, 7-11, 13-17 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Thiagarajan et al. (“Thiagarajan”; IEEE transactions on medical imaging 41, no. 4 (2021): 815-825).
The bold and italicized text below are the limitations of the instant claims, and the italicized text serves to map the prior art onto the instant claims.
Claims 1 and 16:
A method for assessing uncertainty of a histopathological image prediction, the method comprising:
A computer program product for assessing uncertainty of a histopathological image prediction comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
Thiagarajan discloses uncertainty quantified by Bayesian neural network (BNN) classifiers for breast histopathology images (title). Networks were applied on GPUs using Apache MXNet on Python 3, and details of the computational platform are given in the supplementary document (pg. 819, col. 1, para. 5).
sampling a plurality of deep neural network models trained using a plurality of histopathological images;
Thiagarajan discloses that data was sampled from trained the BNN. Thiagarajan recites “[t]he neural network can be defined as a probabilistic model P(y|x,w), where x∈Rp is an input to the network, y∈Υ is the output. Where Υ is a set of all possible outputs. The network consists of a set of trainable random parameters w. This set of parameters w is learned using a complete Bayesian approach. In this approach given training data D, the posterior distribution of weights P(w|D) is calculated using Bayesian inference for neural networks which involves marginalization over all possible values of w. Once the posterior distribution P(w|D) is obtained, predictions on the unseen data are obtained by taking expectations on the predictive distributions” (pg. 817, col. 1, para. 5).
Thiagarajan samples from the variational posterior during training of the Bayesian-CNN (pg. 818, col. 1, para. 3-4). The models were trained on breast histopathological images (pg. 816, col. 2, last para.).
determining an uncertainty in predictions based on the plurality of sampled deep neural network models;
Thiagarajan recites “[t]he uncertainty quantification becomes extremely important when dealing with the applications related to autonomous vehicles, medical imaging, etc. Bayesian deep learning makes it possible to quantify the uncertainties in the prediction as we have probability distribution over weights. Taking an expectation of the predictive posterior probability distribution: (w|θ)[P(yˆ|xˆ,w)] gives us the most probable prediction of the unknown data xˆ. The variance of the predictive posterior probability distribution: Varq(w|θ)[P(y^|x^,w)] quantifies the uncertainties” (pg. 818, col. 1, para. 5).
computing an uncertainty threshold based on the uncertainty in the predictions;
Thiagarajan recites “[f]or every threshold of aleatoric uncertainty, we divide the test data into two subsets having uncertainty lower and higher than the threshold … For other data sets this threshold on uncertainty can be set by looking at change in slope of the accuracy vs uncertainty plot” (pg. 823, col. 1, last para.) (Figure 10) (Figure S5).
categorizing an uncertainty of a histopathological image prediction by comparing an uncertainty associated with the histopathological image prediction with the uncertainty threshold.
Thiagarajan recites “[t]he ratio of the number of images in the low uncertainty subset is plotted against the uncertainty–threshold used to create the subset in Fig. 10(a). The accuracy corresponding to each of this subset are also plotted against the threshold. The number of false-negative and false-positive predictions are plotted for different thresholds of aleatoric uncertainty in Fig. 10(b)” (pg. 822, col. 1, last para.).
Claim 10:
The claim mapping of claim 1 applies to claim 10. The difference between claims 1 and 10 is that claim 10 requires providing an image to a DNN to obtain an image prediction/uncertainty then outputting a pathology in an image based on a comparison. Thiagarajan teaches that the BNN and Bayesian-CNN were trained on breast histopathology images (pg. 816, col. 2, last para.) (Figure 2). Uncertainty for model predictions were estimated (pg. 818, col. 1, para. 5). Figures 8 and 9 show a set of sample images from the test data set, their predicted class and their aleatoric and epistemic uncertainty in prediction (pg. 821, col. 2, last para.).
Claims 2, 11 and 17:
Thiagarajan recites “[i]n Bayesian networks the prior distribution on weights p(w) introduces regularisation of weights automatically. A Gaussian prior yields L2 regularisation on the weights and a Laplace prior yields L1 regularisation on the weights. In addition, due to the stochastic nature of the parameters, an average across multiple models is computed during training which introduces a regularisation effect on the network” (pg. 820, col. 1, last para.).
Thiagarajan recites “[t]he improvement in the performance of modified Bayesian–CNN is due to the adaptive nature of the activation function. The adaptive activation function with a probabilistic learning parameter works better because through it the network trains an ensemble of activation functions where each activation function has its weights drawn from a probability distribution which is learned from the data” (pg. 816, col. 2, para. 3) (pg. 821, col. 1, para. 2).
Claims 4 and 13:
Whole slide images of breast histopathology images were used (pg. 816, col. 2, last para. – pg. 817, col. 1, para. 1). Also, Thiagarajan recites “CNNs were also successfully implemented in problems involving segmentation and detecting regions of interest that contain in-depth discriminatory information for classification in large whole-slide images” (pg. 816, col. 1, para. 1).
Claims 5, 14 and 19:
Thiagarajan recites “[t]he variational posterior (q) is composed of independent Gaussian distribution for each parameter. The sample of weights are obtained by sampling the unit Gaussian, shifting and scaling by the mean μ and a standard deviation σ respectively. To ensure that the standard deviation is always non-negative, it is expressed as σ=softplus(ρ)=log(1+exp(ρ)), point-wise” (pg. 818, col. 1, para. 2).
Claim 7:
Thiagarajan recites “[t]he uncertainty values associated with an individual image denote the confidence with which the network predicts the class label for that image” (pg. 821, col. 2, last para.). Figure 10 shows samples below an uncertainty threshold indicating they have high confidence. Thiagarajan recites “[t]he low uncertainty images have a clear separation between the positive and negative classes in the latent space. Thus the model classifies these images with high confidence” (pg. 823, col. 2, para. 1).
Claim 8:
The BNN and Bayesian-CNN generate predictions for the histopathology images (Figure 4) (Table 3) (pg. 816, col. 2, last para. – pg. 817, col. 1 para. 1).
Claim 9:
Referring to the Bayesian-CNN, Thiagarajan recites “[i]n transfer learning, the learned parameters for one data set are utilized for other data sets to perform a similar task. Here we use the weights of VGG16 architecture trained on Imagenet data set for classification and perform transfer learning to classify the histopathological data” (pg. 820, col. 1, para. 2).
Claim 15:
Claim 15 recites a product by process. This is because the uncertainty threshold in claim 10 recites a product by process as indicated by “the uncertainty threshold having been determined by”. MPEP 2113.I recites “[E]ven though product-by-process claims are limited by and defined by the process, determination of patentability is based on the product itself. The patentability of a product does not depend on its method of production. If the product in the product-by-process claim is the same as or obvious from a product of the prior art, the claim is unpatentable even though the prior product was made by a different process.” As such, the threshold of aleatoric uncertainty of Thiagarajan (pg. 823, col. 1, last para.) reads on the uncertainty threshold of claims 10 and 15, even though it was determined in a different way.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 3, 12 and 18 are rejected under 35 USC 103 for being unpatentable over Thiagarajan et al. (“Thiagarajan”; IEEE transactions on medical imaging 41, no. 4 (2021): 815-825) in view of Rączkowska et al. (“Rączkowska”; NPL ref. no. CB1 on IDS filed 12/29/2023; Scientific reports 9, no. 1 (2019): 14347).
The limitations of claims 1, 10 and 16 are taught above by Thiagarajan in Claim Rejections - 35 USC § 102.
Claims 3, 12 and 18:
Thiagarajan discloses a Bayesian-CNN that contains fully connected layers (pg. 819, col. 2, para. 3). However, the Bayesian-CNN does not contain dropout-enabled hidden layers.
Rączkowska discloses an accurate, reliable and active (ARA) image classification framework and introduces a Bayesian Convolutional Neural Network (ARA-CNN) for classifying histopathology images of colorectal cancer, which outputs uncertainty measurements for each tested image (abstract). ARA-CNN applies dropout to two fully connected layers with 32 units preceding auxiliary and final output (pg. 3, para. 3).
It would have been prima facie obvious to have modified Thiagarajan’s Bayesian-CNN by applying dropout to the fully connected layers of the model as taught by Rączkowska. Motivation for doing so is taught by Rączkowska who teaches that dropout reduces overfitting for the ARA-CNN (pg. 3, para. 2). There would have been a reasonable expectation of success to modify the Bayesian-CNN to contain dropout-enabled hidden layers because Rączkowska demonstrates that a CNN can be modified to contain dropout-enabled fully connected layers (pg. 3, para. 3).
Rejection over Thiagarajan et al. in view of Schisterman et al.
Claims 6 and 20 are rejected under 35 USC 103 for being unpatentable over Thiagarajan et al. (“Thiagarajan”; IEEE transactions on medical imaging 41, no. 4 (2021): 815-825) in view of Schisterman et al. (“Schisterman”; Statistics in medicine 27, no. 2 (2008): 297-315).
The limitations of claims 1 and 16 are taught above by Thiagarajan in Claim Rejections - 35 USC § 102.
Claims 6 and 20:
Thiagarajan calculates an uncertainty threshold (pg. 823, col. 1, last para.) (Figure 10) (Figure S5). However, the uncertainty threshold is not calculated based on maximizing Youden index associated with sensitivity and specificity.
Schisterman teaches “The Youden Index is often used as a summary measure of the receiver operating characteristic curve. It measures the effectiveness of a diagnostic marker and permits the selection of an optimal threshold value or cutoff point for the biomarker of interest” (abstract). Youden index is defined as:
PNG
media_image1.png
131
641
media_image1.png
Greyscale
(pg. 1).
Schisterman teaches “The critical threshold value t*, which achieves this maximum, will be referred to as the ‘optimal’ threshold. The optimal threshold is used as a criterion for classifying subjects as healthy (diseased) if their observed marker value is less than or equal to (greater than) t*” (pg. 2, para. 1).
It would have been prima facie obvious to have calculated the uncertainty threshold of Thiagarajan by maximizing Youden index as taught by Schisterman because Schisterman teaches that “[i]n clinical practice, finding the location of the critical threshold value for discriminating cases and controls with minimal misclassification is of central interest” (pg. 2, para. 3). Thus, one of skill in the art would want an optimal uncertainty threshold in Thiagarajan in order to reduce misclassification. There would have been a reasonable expectation of success because Schisterman states that Youden index is a common measure of the ROC curve and can be used to select an optimal threshold (abstract).
Conclusion
No claims are allowed.
Notable, but not relied upon, prior art includes: Zhao et al. (Automotive Innovation 5, no. 1 (2022): 70-78) and Tomita et al. (Journal of Spacecraft and Rockets 59, no. 6 (2022): 1800-1808) who calculate uncertainty thresholds based on mean value of uncertainty. Linmans et al. (Medical Image Analysis 83 (January 2023): 102655). Joshi et al. (Scientific reports 12, no. 1 (2022): 14628) uses BNN for cancer classification. Ponzio et al. (NPL ref. no. CA1 on IDS filed 12/29/2023; In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1139-1143. IEEE, 2020) computes an automated uncertainty threshold from uncertainty values from each sample. Wu et al. (US 2022/0366214 A1).
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Noah A. Auger whose telephone number is (703)756-4518. The examiner can normally be reached M-F 7:30-4:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Karlheinz Skowronek can be reached at (571) 272-9047. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.A.A./Examiner, Art Unit 1687
/KAITLYN L MINCHELLA/Primary Examiner, Art Unit 1685
Prosecution Insights