DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that does not use the word “means,” and are interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are, where the generic place holder has been underlined and the functional language italicize:
In claim 11, “means for receiving one or more prompts associated with a medical task;
means for encoding at least one of the one or more prompts into a set of features using a feature encoder network of an LLM (large language model);
means for performing the medical task based on the set of features using a decoder network of the LLM;
means for determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM;
and means for outputting the results of the medical task and the uncertainty measure.”
In claim 12, “means for modeling a distribution of a feature space of the LLM with a probability distribution function.”
In claim 15, “means for modeling an in-domain feature space of the LLM;
means for generating an in-domain probability distribution function over the in-domain feature space for the set of features and for a distribution of a feature space of the LLM;
means for modeling an out-of-domain probability distribution function as the complement of the in-domain probability distribution function;
and means for determining whether the set of features is out-of-domain of the LLM based on the in-domain probability distribution function and the out-of-domain probability distribution function.”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4 and 11-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 and analogous claim 14 recites the limitation "the medical task images.” There is insufficient antecedent basis for this limitation in the claim.
In claim 11, “means for receiving one or more prompts associated with a medical task;
means for encoding at least one of the one or more prompts into a set of features using a feature encoder network of an LLM (large language model);
means for performing the medical task based on the set of features using a decoder network of the LLM;
means for determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM;
and means for outputting the results of the medical task and the uncertainty measure.”
In claim 12, “means for modeling a distribution of a feature space of the LLM with a probability distribution function.”
In claim 15, “means for modeling an in-domain feature space of the LLM;
means for generating an in-domain probability distribution function over the in-domain feature space for the set of features and for a distribution of a feature space of the LLM;
means for modeling an out-of-domain probability distribution function as the complement of the in-domain probability distribution function;
and means for determining whether the set of features is out-of-domain of the LLM based on the in-domain probability distribution function and the out-of-domain probability distribution function.”
The above limitations have been evaluated under the three-prong test set forth in MPEP § 2181, subsection I, but the result is inconclusive. Thus, it is unclear whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because MPEP 2181 (II)(B) discloses “ …To claim a means for performing a specific computer-implemented function and then to disclose only a general purpose computer as the structure designed to perform that function amounts to pure functional claiming… The corresponding structure is not simply a general purpose computer by itself but the special purpose computer as programmed to perform the disclosed algorithm. Aristocrat, 521 F.3d at 1333, 86 USPQ2d at 1239. Thus, the specification must sufficiently disclose an algorithm to transform a general purpose microprocessor to the special purpose computer. See Aristocrat, 521 F.3d at 1338, 86 USPQ2d at 1241…Accordingly, a rejection under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph is appropriate if the specification discloses no corresponding algorithm associated with a computer or microprocessor. Aristocrat, 521 F.3d at 1337-38, 86 USPQ2d at 1242.”. The boundaries of this claim limitation are ambiguous; therefore, the claims are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Claims 12-15 are further rejected on virtue of their dependency to claim 11.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 11-15 are further rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
In regards to claims 11, 12 and 15, when a claim containing a computer-implemented 35 U.S.C. 112(f) claim limitation is found to be indefinite under 35 U.S.C. 112(b) for failure to disclose sufficient corresponding structure (e.g., the computer and the algorithm) in the specification that performs the entire claimed function, it will also lack written description under 35 U.S.C. 112(a). See MPEP § 2163.03, subsection VI.
Claims 12-15 are further rejected on virtue of their dependency to claim 11.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 and 16-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In regards to claim 1,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A – Prong 1: Judicial Exception Recited?
MPEP 2106.04(a)(2)(II) “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
Further, the MPEP recites “It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018) (holding that claims to a ‘‘series of mathematical calculations based on selected information’’ are directed to abstract ideas); Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a ‘‘process of organizing information through mathematical correlations’’ are directed to an abstract idea); and Bancorp Servs., LLC v. Sun Life Assurance Co. of Can. (U.S.), 687 F.3d 1266, 1280, 103 USPQ2d 1425, 1434 (Fed. Cir. 2012) (identifying the concept of ‘‘managing a stable value protected life insurance policy by performing calculations and manipulating the results’’ as an abstract idea).”
Yes, the claim recites a mathematical concept, specifically:
determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM
Examiner’s note: Examiner interprets the limitation in light of the specification wherein the uncertainty measure may be a probability and the uncertainty quantification module is a distribution model (ie a mathematical model) (“[0027] The uncertainty measure may comprise, e.g., a context confidence score and/or an answer confidence score. The context confidence score (e.g., for few-shot learning) represents the probability of the provided contextual information (received in the one or more prompts) belonging to the in-domain feature space of the LLM…”) (“[0028] In one example, as shown in workflow 200 of Figure 2, the uncertainty quantification module is distribution model 212, which generates uncertainty measures 214-A and 214-B (collectively referred to as uncertainty measures 214) based on set of features 206...”)
Thus, the limitation encompasses calculating a probability using a mathematical model based on the results of the medical tasks and the set of features.
Therefore, the claim recites a mathematical concept.
MPEP 2106.04(a)(2)(I) “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
Yes, the claim recites a mental process, specifically:
encoding at least one of the one or more prompts into a set of features
This limitation encompasses an evaluation of a prompt into a set of features. For example, one of ordinary skills in the art would be able to receive a sentence and calculate linear projections on the prompt.
performing the medical task based on the set of features
This limitation encompasses providing an opinion (ex a summary of the input) with an evaluation of the set of features.
Therefore, the claim recites a mental process.
Step 2A – Prong 2: Integrated into a Practical Solution?
MPEP 2106.05(f) Mere Instructions To Apply An Exception has found simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. The following steps are mere instructions to apply:
A computer-implemented method (generic computer to apply the abstract idea)
using a feature encoder network of an LLM (large language model) (mere instructions to apply an exception, because they recite no more than an idea of a solution or outcome)
Examiner’s note: Examiner interprets the limitation in light of the specification wherein the encoder may be any “suitable architecture” (“[0023] At step 104 of Figure 1, at least one of the one or more prompts are encoded into a set of features using a feature encoder network of an LLM. The feature encoder network may be of any suitable architecture. The feature encoder network receives as input the at least one of the one or more prompts and generates as output the set of features. The set of features is a feature vector representing low level latent features or embeddings of the at least one of the one or more prompts. The feature space typically has a fixed size, which may be configurable as a parameter (i.e., number of tokens). In one example, as shown in workflow 200 of Figure 2, the feature encoder network is LLM feature encoder 204 for encoding prompts 202-A to generate set of features 206.”)
using a decoder network of the LLM (mere instructions to apply an exception, because they recite no more than an idea of a solution or outcome)
Examiner’s note: Examiner interprets the limitation in light of the specification wherein the decoder may be any “suitable architecture” (“[0025] At step 106 of Figure 1, the medical task is performed based on the set of features using a decoder network of the LLM. The decoder may be of any suitable architecture. The decoder network receives as input the set of features and decodes the set of features to generate as output results of the medical task. In one example, as shown in workflow 200 of Figure 2, the decoder network is decoder network 208, which receives as input the set of features 206 and generates as output results 210 of the medical task. Results 210 comprises an answer to the question of “is cardiomegaly present?” received via prompt 202-B.”)
MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering to be insignificant extra-solution activity. The following steps are insignificant extra-solution activities:
Mere data gathering:
receiving one or more prompts associated with a medical task (receiving data)
Examiner’s note: Examiner interprets the limitation in light of the specification wherein the prompt is user provided input (“[0020] At step 102 of Figure 1, one or more prompts associated with a medical task are received. A prompt is a user provided input to an LLM from which the LLM is to perform the medical task. The prompt may include, for example, text-based instructions, questions, contextual information, and/or any other type of user input. The one or more prompts may be received from a computing device (e.g., computer 502 of Figure 5) with which a user interacting.”)
outputting the results of the medical task and the uncertainty measure (transmitting data)
The additional elements have been considered both individually and as an ordered combination in to determine whether they integrate the exception into a practical application. Therefore, no meaningful limits are imposed on practicing the abstract idea.
The claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitation is mere instructions to apply and mere data gathering (Insignificant Extra-Solution Activity) and a generic device do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
A claim that generically recites an effect of the judicial exception or claims every mode of accomplishing that effect, amounts to a claim that is merely adding the words "apply it" to the judicial exception.
Further, the claim recites receiving and transmitting data by a generic device.
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to
gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary
computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607,
610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP
Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)
(sending messages over a network); buy SAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112
USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);
but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106
(Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how
interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides
the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
(emphasis added)).
The additional elements have been considered both individually and as an ordered
combination in the significantly more consideration.
The claim is ineligible.
In regards to claim 2,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM comprises: modeling a distribution of a feature space of the LLM with a probability distribution function.
This limitation directs to a mathematical calculation with a probability distribution function. See MPEP 2106.04(a)(2)(I)(C.)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
In regards to claim 3,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein the probability distribution function comprises one of a Gaussian mixture model, a kernel density estimate of a Gaussian Process, or inducing points of a Gaussian Process.
This limitation directs to a mathematical calculation with a Gaussian mixture model. See MPEP 2106.04(a)(2)(I)(C.)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
In regards to claim 4,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein the probability distribution function is computed over features of the LLM and image features
This limitation directs to a mathematical calculation of the pdf. See MPEP 2106.04(a)(2)(I)(C.)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
using a pre-trained model
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
image features extracted from the medical task images
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
using a pre-trained model
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
image features extracted from the medical task images
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(iv): Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
In regards to claim 5,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM comprises: modeling an in-domain feature space of the LLM; generating an in-domain probability distribution function over the in-domain feature space for the set of features and for a distribution of a feature space of the LLM; modeling an out-of-domain probability distribution function as the complement of the in-domain probability distribution function;
This limitation directs to mathematical calculations with probability distribution functions. See MPEP 2106.04(a)(2)(I)(C.)
and determining whether the set of features is out-of-domain of the LLM based on the in-domain probability distribution function and the out-of-domain probability distribution function
This limitation directs to a mental process that can be performed in the human mind, by a human using pen and paper, or using a computer as a tool to perform the concept and encompasses providing an evaluation of the mathematical calculations to provide a judgement on whether the set of features is out-of-domain. See MPEP 2106.04(a)(2)(III)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
In regards to claim 6,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
transmitting a notification to a user to revise the one or more prompts based on the uncertainty measure.
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
transmitting a notification to a user to revise the one or more prompts based on the uncertainty measure.
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
In regards to claim 7,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
restricting the results of the medical task based on the uncertainty measure
This limitation directs to a mental process that can be performed in the human mind, by a human using pen and paper, or using a computer as a tool to perform the concept and encompasses an evaluation of the previously calculated uncertainty measure to provide a judgement on the results of the medical tasks to restrict. See MPEP 2106.04(a)(2)(III)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
In regards to claim 8,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein the uncertainty measure comprises at least one of a context confidence score or an answer confidence score
This limitation directs to a mathematical calculation. See MPEP 2106.04(a)(2)(I)(C.)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
In regards to claim 9,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the LLM is constrained to a specific medical domain
This limitation merely indicates a field of use or technological environment in which the judicial exception is performed. This type of limitation merely confines the use of the abstract idea to a particular field of use and thus fails to add an inventive concept to the claims. See MPEP § 2106.05(h)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the LLM is constrained to a specific medical domain
This limitation merely indicates a field of use or technological environment in which the judicial exception is performed. This type of limitation merely confines the use of the abstract idea to a particular field of use and thus fails to add an inventive concept to the claims. See MPEP § 2106.05(h)
In regards to claim 10,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the medical task comprises at least one of summarizing one or more medical reports, determining a patient condition, and radiology reading assistance
This limitation merely indicates a field of use or technological environment in which the judicial exception is performed. This type of limitation merely confines the use of the abstract idea to a particular field of use and thus fails to add an inventive concept to the claims. See MPEP § 2106.05(h)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the medical task comprises at least one of summarizing one or more medical reports, determining a patient condition, and radiology reading assistance
This limitation merely indicates a field of use or technological environment in which the judicial exception is performed. This type of limitation merely confines the use of the abstract idea to a particular field of use and thus fails to add an inventive concept to the claims. See MPEP § 2106.05(h)
Claim 16 (machine) is rejected on the same grounds under 35 U.S.C. 101 as claim 1 as they are substantially similar, respectively, Mutatis mutandis.
Claim 17 (machine) is rejected on the same grounds under 35 U.S.C. 101 as claim 6 as they are substantially similar, respectively, Mutatis mutandis.
Claim 18 (machine) is rejected on the same grounds under 35 U.S.C. 101 as claim 7 as they are substantially similar, respectively, Mutatis mutandis.
Claim 19 (machine) is rejected on the same grounds under 35 U.S.C. 101 as claim 8 as they are substantially similar, respectively, Mutatis mutandis.
Claim 20 (machine) is rejected on the same grounds under 35 U.S.C. 101 as claim 9 as they are substantially similar, respectively, Mutatis mutandis.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 4-12 and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub No. US20240289558A1 Muraoka et al. (“Muraoka”) in view of US Pub no. US20180341752A1 Bernard et al. (“Bernard”).
In regards to claim 1 and analogous claims 11 and 16,
Muraoka teaches A computer-implemented method comprising: receiving one or more prompts [associated with a medical task];
PNG
media_image1.png
679
548
media_image1.png
Greyscale
(Muraoka, fig. 4 410, “[0014] In yet another aspect of the invention, yet another method for performing a downstream task with a language model is provided. The method includes: obtaining a dataset for the downstream task, the dataset having at least a training set and a testing set; constructing a datastore by applying the language model to the training set; applying the language model to a prompt-applied sentence prompt(x) [receiving one or more prompts] from an instance x in the testing set to predict a label y…”)
Muraoka teaches encoding at least one of the one or more prompts into a set of features using a feature encoder network of an LLM (large language model);
Examiner’s note: Examiner interprets the limitation in light of the specification wherein the encoder may be any “suitable architecture” (“[0023] At step 104 of Figure 1, at least one of the one or more prompts are encoded into a set of features using a feature encoder network of an LLM. The feature encoder network may be of any suitable architecture.”)
(Muraoka, fig. 4 418, “[0061] In step 418, the large language model feature vector hLM(prompt(x)) is extracted [encoding at least one of the one or more prompts into a set of features ie feature vector using a feature encoder network of an LLM (large language model)]. As described in detail above, the feature vector hLM(prompt(x)) is part of a triplet obtained by feeding prompt(x) to the large language model.”)
Muraoka teaches performing the [medical] task based on the set of features using a decoder network of the LLM;
Examiner’s note: Examiner interprets the limitation in light of the specification wherein the decoder may be any “suitable architecture” (“[0025] At step 106 of Figure 1, the medical task is performed based on the set of features using a decoder network of the LLM. The decoder may be of any suitable architecture. The decoder network receives as input the set of features and decodes the set of features to generate as output results of the medical task. In one example, as shown in workflow 200 of Figure 2, the decoder network is decoder network 208, which receives as input the set of features 206 and generates as output results 210 of the medical task. Results 210 comprises an answer to the question of “is cardiomegaly present?” received via prompt 202-B.”)
(Muraoka, fig. 4. 428, [0068], “Namely, the k-Nearest Neighbor search results explicitly show which instance, i.e., sentences, from the datastore the large language model considers to be close to the input sentence prompt(x). To do so, in addition to outputting the final prediction {circumflex over (p)}(y|prompt(x)) to a user, in step 430 the results from the k-Nearest Neighbor search of the datastore (see step 420, described above) also output to the user by system 200 as a way to explain the prediction of the large language model [performing the … task based on the set of features using a decoder network of the LLM; wherein performing the task is providing the response (final prediction) to the prompt x].”)
Muraoka teaches determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM;
Examiner interprets the limitation in light of the specification wherein the uncertainty score may be based on a feature similarity (“[0027] The uncertainty measure may comprise, e.g., a context confidence score and/or an answer confidence score. The context confidence score (e.g., for few-shot learning) represents the probability of the provided contextual information (received in the one or more prompts) belonging to the in-domain feature space of the LLM. This would indicate if the medical task was intended by the training constraints. The context confidence score may be based on the feature similarity between the question and contextual information provided in the one or more prompts and the results of the medical task.”)
(Muraoka, fig. 4. 416 and 426, “[0063] Specifically, a feature space 422 (i.e., the datastore) is shown which is built on the feature vectors htrain obtained from the large language model and the Positive and Negative training instances (i.e., circles with a diamond-shaped pattern and un-patterned circles, respectively) closest to the testing instance hLM(prompt(x)) aka the query vector (shown as the circle with a dotted pattern). The distance between the circles illustrates the similarity between the instances, regardless of which set, training or testing set, an instance originates from. According to the present techniques, the k-Nearest Neighbor (kNN) search result, i.e., the k closest instances to the query vector hLA (prompt(x)), will be used to compute a probability distribution {circumflex over (p)}kNN [determining an uncertainty measure ie KNN search results ({circumflex over (p)}kNN) associated with results of the … task based on the set of features using an uncertainty quantification module of the LLM].”)
Muraoka teaches and outputting the results of the [medical] task and the uncertainty measure.
(Muraoka, fig. 4 430, [0068], “Namely, the k-Nearest Neighbor search results explicitly show which instance, i.e., sentences, from the datastore the large language model considers to be close to the input sentence prompt(x). To do so, in addition to outputting the final prediction {circumflex over (p)}(y|prompt(x)) to a user, in step 430 the results from the k-Nearest Neighbor search of the datastore (see step 420, described above) also output to the user by system 200 as a way to explain the prediction of the large language model [outputting the results of the … task ie final prediction and the uncertainty measure ie KNN search results].”)
However, Muraoka does not explicitly teach prompts associated with a medical task
Bernard teaches prompts associated with a medical task
(Bernard, “[0101] FIG. 8A presents an embodiment of the medical scan assisted review system 102. The medical scan assisted review system 102 can be used to aid medical professionals or other users in diagnosing, triaging, classifying, ranking, and/or otherwise reviewing medical scans by presenting a medical scan for review by a user by transmitting medical scan data of a selected medical scan and/or interface feature data of selected interface features of to a client device 120 corresponding to a user of the medical scan assisted review system for display via a display device of the client device. The medical scan assisted review system 102 can generate scan review data 810 for a medical scan based on user input to the interactive interface 275 [prompts associated with a medical task] displayed by the display device in response to prompts to provide the scan review data 810, for example, where the prompts correspond to one or more interface features.”)
Muraoka is considered to be analogous to the claimed invention because they are in the same field of large language models. Bernard is considered to be analogous to the claimed invention because they are in the same field of applying machine learning/AI to the medical field. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Muraoka to incorporate the teachings of Bernard in order to apply the architecture of Muraoka to the medical scan system of Bernard to improve the performance of the medical scan assisted review system wherein the architecture of Muraoka provides improved interpretability (Bernard, [0153], “The medical scan assisted review system 102 automatically generates scan review data 810 corresponding to the new finding based on the text entered by the user in the text window and the polygon indicating the region of interest, automatically determined based on the five vertices 850 indicated by the user, as presented in the interactive interface 275 of FIG. 8R in response to the user electing to approve the new finding, and this scan review data 810 corresponding to the new finding can be added to the medical scan database and/or utilized in training sets used to improve the performance of the medical scan assisted review system 102 or other subsystems in subsequent uses.”) (Muraoka, “[0009] Therefore, techniques for fine-tuning free evaluation of large language models that improve the performance in downstream tasks over current methods, and which also increase the interpretability of the large language model predictions would be desirable.”)
In regards to claim 2 and analogous claim 12,
Muraoka and Bernard teaches The computer-implemented method of claim 1, Muraoka teaches wherein determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM comprises: modeling a distribution of a feature space of the LLM with a probability distribution function.
(Muraoka, “[0063] Specifically, a feature space 422 (i.e., the datastore) is shown which is built on the feature vectors htrain obtained from the large language model and the Positive and Negative training instances (i.e., circles with a diamond-shaped pattern and un-patterned circles, respectively) closest to the testing instance hLM(prompt(x)) aka the query vector (shown as the circle with a dotted pattern). The distance between the circles illustrates the similarity between the instances, regardless of which set, training or testing set, an instance originates from. According to the present techniques, the k-Nearest Neighbor (kNN) search result, i.e., the k closest instances to the query vector hLA (prompt(x)), will be used to compute a probability distribution {circumflex over (p)}kNN [modeling a distribution of a feature space of the LLM with a probability distribution function].”)
(Muraoka, “[0067] It is notable however, that the present techniques are more generally applicable to the combination of any output probability distribution from the large language model with the probability distribution {circumflex over (p)}kNN, including the (pre-debiased) output distribution PLM(y∈V|prompt(x)) (from step 414)”)
In regards to claim 4 and analogous claim 14,
Muraoka and Bernard teaches The computer-implemented method of claim 2, Muraoka teaches wherein the probability distribution function is computed over features of the LLM
(Muraoka, “[0063] Specifically, a feature space 422 (i.e., the datastore) is shown which is built on the feature vectors htrain obtained from the large language model and the Positive and Negative training instances (i.e., circles with a diamond-shaped pattern and un-patterned circles, respectively) closest to the testing instance hLM(prompt(x)) aka the query vector (shown as the circle with a dotted pattern). The distance between the circles illustrates the similarity between the instances, regardless of which set, training or testing set, an instance originates from. According to the present techniques, the k-Nearest Neighbor (kNN) search result, i.e., the k closest instances to the query vector hLA (prompt(x)), will be used to compute a probability distribution {circumflex over (p)}kNN [wherein the probability distribution function is computed over features of the LLM].”)
Muraoka discloses the present technique is generally applicable to a combination of any output probability distribution from the LLM (Muraoka, “[0067] It is notable however, that the present techniques are more generally applicable to the combination of any output probability distribution from the large language model with the probability distribution {circumflex over (p)}kNN, including the (pre-debiased) output distribution PLM(y∈V|prompt(x)) (from step 414)”)
Bernard teaches and image features extracted from the medical task images using a pre-trained model.
(Bernard, “[0102] In various embodiments, the medical scan assisted review system 102 is operable to receive, via a network, a medical scan for review. Abnormality annotation data 442 is generated by identifying one or more of abnormalities in the medical scan by utilizing a computer vision model that is trained on a plurality of training medical scans [and image features extracted from the medical task images using a pre-trained model]. The abnormality annotation data 442 includes location data and classification data for each of the plurality of abnormalities and/or data that facilitates the visualization 825 of the abnormalities in the scan image data 410…
[0267] In various embodiments, a probability distribution function (PDF) can be used to determine the center location of the subregion, where selecting a subregion that is closer to the center to the subregion is more probable than selecting a subregion that is further from the center of the subregion.”
PNG
media_image2.png
424
662
media_image2.png
Greyscale
)
In regards to claim 5 and analogous claim 15,
Muraoka and Bernard teaches The computer-implemented method of claim 1, Muraoka teaches wherein determining an uncertainty measure associated with results of the medical task based on the set of features using an uncertainty quantification module of the LLM comprises: modeling an in-domain feature space of the LLM;
Examiner’s note: Examiner interprets in/out-domain in light of the specification wherein in-domain appears to be training data and out-domain appears to be testing data ([0029], “The in-domain feature space corresponds to features of data on which the LLM is trained to perform a medical task and the out-of-domain feature space corresponds to features of data on which the LLM is not trained to perform a medical task.”)
(Muraoka, “[0063] Specifically, a feature space 422 (i.e., the datastore) is shown which is built on the feature vectors htrain obtained from the large language model [modeling an in-domain feature space of the LLM] and the Positive and Negative training instances (i.e., circles with a diamond-shaped pattern and un-patterned circles, respectively) closest to the testing instance hLM(prompt(x)) aka the query vector (shown as the circle with a dotted pattern)…”)
Muraoka teaches generating an in-domain probability distribution function over the in-domain feature space for the set of features and for a distribution of a feature space of the LLM;
(Muraoka, [0063], “…The distance between the circles illustrates the similarity between the instances, regardless of which set, training or testing set, an instance originates from. According to the present techniques, the k-Nearest Neighbor (kNN) search result, i.e., the k closest instances to the query vector hLA (prompt(x)), will be used to compute a probability distribution {circumflex over (p)}kNN [generating an in-domain probability distribution function over the in-domain feature space for the set of features and for a distribution of a feature space of the LLM].”)
Muraoka teaches modeling an out-of-domain probability distribution function as the complement of the in-domain probability distribution function;
(Muraoka, “[0065] In step 428, a final prediction {circumflex over (p)}(y|prompt(x)) of the large language model is computed by combining the output probability distribution from the large language model and the probability distribution {circumflex over (p)}kNN [modeling an out-of-domain probability distribution function ie output probability distribution from the large language model (test dataset) as the complement of the in-domain probability distribution function ie {circumflex over (p)}kNN (training dataset)] computed of the k-Nearest Neighbor search results.”)
Muraoka teaches and determining whether the set of features is out-of-domain of the LLM based on the in-domain probability distribution function and the out-of-domain probability distribution function.
(Muraoka, “[0065] In step 428, a final prediction {circumflex over (p)}(y|prompt(x)) of the large language model is computed by combining the output probability distribution from the large language model and the probability distribution {circumflex over (p)}kNN [determining whether the set of features is out-of-domain of the LLM based on the in-domain probability distribution function and the out-of-domain probability distribution function; wherein the set of features is considered out-of-domain if it is in the output probability distribution from the large language model] computed of the k-Nearest Neighbor search results.”)
In regards to claim 6 and analogous claim 17,
Muraoka and Bernard teaches The computer-implemented method of claim 1,
Bernard teaches further comprising: transmitting a notification to a user to revise the one or more prompts based on the uncertainty measure.
(Bernard, [0216], “The interface can prompt the user to indicate the appropriate scan category 1120 and/or prompt the user to confirm and/or edit the inferred scan category [transmitting a notification to a user to revise the one or more prompts], also presented to the user [based on the uncertainty measure; wherein the uncertainty measure is the combined probability distributions output and explanation of Muraoka]. For example, scan review data 810 can be automatically generated to reflect the user generated and/or verified scan category 1120, This user indicated scan category 1120 can be utilized to select to the medical scan inference function 1105 and/or to update the scan classifier data 420 or other metadata accordingly.”)
In regards to claim 7 and analogous claim 18,
Muraoka and Bernard teaches The computer-implemented method of claim 1,
Muraoka teaches further comprising: restricting the results of the medical task based on the uncertainty measure.
(Muraoka, [0063], “For example, when k=4, the four closest instances to the query vector hLM(prompt(x)) are used as shown in FIG. 4 [restricting the results of the medical task based on the uncertainty measure; wherein the number of closest instances can be restricted] , which are inside the dashed circle 424. In the present example, there are three positive instances and one negative instance as the k-Nearest Neighbor instances (k=4).”)
In regards to claim 8 and analogous claim 19,
Muraoka and Bernard teaches The computer-implemented method of claim 1, Muraoka teaches wherein the uncertainty measure comprises at least one of a context confidence score or an answer confidence score.
Examiner interprets the limitation in light of the specification wherein the uncertainty score may be based on a feature similarity (“[0027] The context confidence score may be based on the feature similarity between the question and contextual information provided in the one or more prompts and the results of the medical task.”)
(Muraoka, fig. 4. 416 and 426, “[0063] Specifically, a feature space 422 (i.e., the datastore) is shown which is built on the feature vectors htrain obtained from the large language model and the Positive and Negative training instances (i.e., circles with a diamond-shaped pattern and un-patterned circles, respectively) closest to the testing instance hLM(prompt(x)) aka the query vector (shown as the circle with a dotted pattern). The distance between the circles illustrates the similarity between the instances, regardless of which set, training or testing set, an instance originates from. According to the present techniques, the k-Nearest Neighbor (kNN) search result, i.e., the k closest instances to the query vector hLA (prompt(x)), will be used to compute a probability distribution {circumflex over (p)}kNN [context confidence score].”)
In regards to claim 9 and analogous claim 20,
Muraoka and Bernard teaches The computer-implemented method of claim 1, Bernard teaches wherein the LLM is constrained to a specific medical domain.
(Bernard, “[0062] Alternatively or in addition, the diagnosis data 440 can include natural language text data 448 annotating or otherwise describing the medical scan as a whole, and/or the abnormality annotation data 442 can include natural language text data 448 annotating or otherwise describing each corresponding abnormality [wherein the LLM is constrained to a specific medical domain]. In some embodiments, some or all of the diagnosis data 440 is presented only as natural language text data 448. In some embodiments, some or all of the diagnosis data 440 is automatically generated by one or more subsystems based on the natural language text data 448, for example, without utilizing the medical scan image data 410, for example, by utilizing one or more medical scan natural language analysis functions trained by the medical scan natural language analysis system 114. Alternatively or in addition, some embodiments, some or all of the natural language text data 448 is generated automatically based on other diagnosis data 440 such as abnormality annotation data 442, for example, by utilizing a medical scan natural language generating function trained by the medical scan natural language analysis system 114.”)
In regards to claim 10,
Muraoka and Bernard teaches The computer-implemented method of claim 1, Bernard teaches wherein the medical task comprises at least one of summarizing one or more medical reports, determining a patient condition, and radiology reading assistance.
(Bernard, “[0102] In various embodiments, the medical scan assisted review system 102 is operable to receive, via a network, a medical scan for review. Abnormality annotation data 442 is generated by identifying one or more of abnormalities in the medical scan by utilizing a computer vision model that is trained on a plurality of training medical scans. The abnormality annotation data 442 includes location data and classification data for each of the plurality of abnormalities and/or data that facilitates the visualization 825 of the abnormalities in the scan image data 410. Report data 830 including text describing each of the plurality of abnormalities is generated based on the abnormality data [summarizing one or more medical reports]. The visualization 825 and the report data 830 (collectively displayed annotation data 820) is transmitted to a client device.”)
Claim(s) 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Muraoka in view of Bernard in further view of Pearce, Tim, Alexandra Brintrup, and Jun Zhu. "Understanding softmax confidence and uncertainty." arXiv preprint arXiv:2106.04972 (2021) (“Pearce”).
In regards to claim 3 and analogous claim 13,
Muraoka and Bernard teaches The computer-implemented method of claim 2, Pearce teaches wherein the probability distribution function comprises one of a Gaussian mixture model, a kernel density estimate of a Gaussian Process, or inducing points of a Gaussian Process.
(Pearce, Section 2., “A probability density is estimated, ˆq(z) ≈ pin(z), and the negative log likelihood is used as an uncertainty score (again higher is more uncertain),
PNG
media_image3.png
26
428
media_image3.png
Greyscale
In this work we use a Gaussian mixture model (GMM) with K (equal to the number of classes) components to estimate this density (though not restricted to one component per class, and with no restrictions on the covariance structure).”)
Pearce is considered to be analogous to the claimed invention because they are in the same field of determining uncertainty in neural networks. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Muraoka and Bernard to incorporate the teachings of Pearce in order to interpret the probability distribution output of the LLM in terms of uncertainty and a gaussian mixture model allows analytical integration of class clusters (Pearce, Section 1., “For example, out-of-distribution (OOD) detection requires predicting if a data point is from the training distribution or not (fig. 1 contains example distributions), where success depends on good epistemic uncertainty estimates. A system incapable of capturing epistemic uncertainty should do no better than random guessing– an AUROC of 50%. Yet directly interpreting softmax confidence as uncertainty can score from 75% to 99% across datasets and architectures (section B.1). Meanwhile, adding modifications purposefully designed to capture uncertainty typically improves AUROC by just 1% to 5%. This suggests that softmax confidence may be more useful as an indicator of epistemic uncertainty than widely thought.”) (Pearce, Section 3.1, “Making the assumption that final-layer features in the training data follow a mixture of Gaussians allows analytical integration of class clusters in corollary 1.1.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US20160267226A1 Xu et al. teaches A system for correlating patient radiology and pathology reports to track discordance among radiology and pathology diagnoses includes a natural language processor engine which extracts radiological information and pathology information. A correlation module correlates the radiology information and pathology information in a specific time period. A visualization graphical user interface indicates the correlation of radiology information and pathology information in a patient history. A tracking module which tracks misdiagnosis cases.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights