DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of this application being a National Stage application of the PCT Application No. PCT/JP2023/013968, filed on April 4, 20223. As well as acknowledgement of priority to JP 2022-120852 with filing date of July 28, 2022, acknowledged under 35 USC 119(a)-(d) or (f).
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on 12/23/2024 has been reviewed and the listed references have been considered.
Drawings
The 9-page drawings have been considered and placed on record in the file.
Status of Claims
Claims 1-9 are pending.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpretated under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are, “determination evaluation device …” “uncertainty calculation unit…” analysis evaluation unit…”, and “evaluation result display unit …” in claims 1-9.
Because of these claim limitations being interpretated under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101, based on abstract idea. The claims recite a system and method for evaluating a result from a machine learning model, the evaluation consist of calculating uncertainty and further analyzing the uncertainty values. With respect to independent device claim 1:
STEP 1: Do the claims fall within one of the statutory categories?
YES. Claim 1 is directed to a device i.e., a system or a machine.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
YES, the claims are directed toward a mental process (i.e., abstract idea).
The limitation “evaluates a determination result for input data based on a determination” as drafted, recite an abstract idea, such as a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind of a person, i.e., concepts performed in the human mind (including observation, evaluation, judgement, opinion).
As such, a person can review any result and evaluate the result or an object with a degree of error or lack thereof either mentally or using a pen and paper. The mere nominal recitation that the various steps are being executed by a processor (e.g., processing unit) does not take the limitations out of the mental process grouping.
The limitation “calculates uncertainty of the determination result” and “evaluates the determination result on a basis of the uncertainty” as drafted recite an abstract idea, such as a process that, under the broadest reasonable interpretation, covers performance of the limitation using mathematic concepts. As such, calculations and evaluations on the basis of a calculated result is the application of applying mathematical concepts.
Thus, the claims recite an abstract idea.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
The only additional elements “determination evaluation device”, “uncertainty calculation unit”, and “analysis evaluation unit” are recited at a high level of generality and merely equate to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). See also MPEP 2106.04(a)(2)(III) with respect to Mental Processes: “Nor do the courts distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer”. See also MPEP 2106.04(a)(2)(III)(C)(3) Using a computer as tool to perform a mental process and MPEP 2106.04(a)(2)(III)(D) as well as the case law cited therein.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
NO,
The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception, because as discussed above with respect to integration of the abstract idea into practical application, the additional step/element/limitation amounts to no more than an abstract idea performed on a computer. The additional elements are simply appending well-understood routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC) per MPEP 2106.05(d) and 2106.07(a)(III). Therefore, claim 1 is not patent eligible.
In addition, the elements of claim 8 is analyzed in the same manner as claim 1. Therefore independent claims 1 an d8 are not patent eligible, either.
Similar analysis is made for the dependent claims 2-7, under their broadest reasonable interpretation are identified as: being either directed towards mere data gathering or an abstract idea, mental process and mathematical calculation, and not reciting additional elements that integrate the judicial exception into a practical application, and not reciting additional elements that amount to significantly more than the judicial exception.
For all of the above reasons, claims 1-8 are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, claims 1-8 are not eligible subject matter under 35 U.S.C 101.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter, because the claim recites “determination evaluation program …”. Computer programs, per se, are not in one of the statutory categories of invention because a computer program is merely a set of instructions capable of being executed by a computer - the computer program itself is not a process. MPEP § 2106.
A computer program, at best, is a functional descriptive material per se. Descriptive material can be characterized as either "functional descriptive material" or "nonfunctional descriptive material." Both types of "descriptive material" are nonstatutory when claimed as descriptive material per se, 33 F.3d at 1360, 31 USPQ2d at 1759. When functional descriptive material is recorded on some computer-readable medium, it becomes structurally and functionally interrelated to the medium and will be statutory in most cases since use of technology permits the function of the descriptive material to be realized. Compare In re Lowry, 32 F.3d 1579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. Cir. 1994) )(discussing patentable weight of data structure limitations in the context of a statutory claim to a data structure stored on a computer readable medium that increases computer efficiency) and >In re Warmerdam, 33 F.3d *>1354, 1360- 61,31 USPQ2d *>1754, 1759 (claim to computer having a specific data structure stored in memory held statutory product-by-process claim) with Warmerdam, 33 F.3d at 1361,31 USPQ2d at 1760 (claim to a data structure per se held nonstatutory). See MPEP 2106.01.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 4, and 6-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mavroeidis et al. (US 2022/0180516 A1).
Regarding claim 1, Mavroeidis teaches “A determination evaluation device that evaluates a determination result (Mavroeidis paragraph [0049] "The present invention provides a method, computer program and processing system for identifies boundaries of lesions within image data") for input data based on a determination model (Mavroeidis paragraph [0049] "The image data is processed using a machine learning algorithm to generate probability data and uncertainty data. The probability data provides, for each image data point of the image data, a probability data points indicating a probability that said image data point is part of a lesion. The uncertainty data provides, for each probability data point, an uncertainty data point indicating an uncertainty of the said probability data point. The uncertainty data is processed to identify or correct boundaries of the lesions"), the determination evaluation device comprising:
an uncertainty calculation unit (Mavroeidis paragraph [0122] "the machine-learning algorithm is a Bayesian deep learning segmentation algorithm, such as a Bayesian neural network. Bayesian-based machine-learning models provide a relatively simple method of calculating the uncertainty of a probability data point, as uncertainty calculation is built into the Bayesian methodology") that calculates uncertainty of the determination result (Mavroeidis paragraph [0062] "The uncertainty data 24 comprises uncertainty data points. Each uncertainty data point corresponds to a respective probability data point (and therefore a respective image data point). Each uncertainty data point is indicative of a level of uncertainty of the indicated probability, e.g. a measure or value representing a (un)certainty that the probability data point is correct. Thus, there are a same number of uncertainty data points as there are probability data points, and therefore of image data point"); and
an analysis evaluation unit that evaluates the determination result on a basis of the uncertainty (Mavroeidis paragraph [0080-0081] "if a magnitude of a neighboring image data point is above an inclusion threshold value, then that neighboring image data point is included in the potential lesion. One approach to implement this could be to change the inclusion threshold based on the level of uncertainty for the same image data point or an average uncertainty of the associated predicted lesion. The thresholds can be parameters that are tuned during the hyperparameter optimization of the segmentation model").”
Regarding claim 2, Mavroeidis teaches “The determination evaluation device according to claim 1, wherein the input data is an image,
the determination is processing of classifying the image into regions of a plurality of classes (Mavroeidis paragraph [0049] "The image data is processed using a machine learning algorithm to generate probability data and uncertainty data. The probability data provides, for each image data point of the image data, a probability data points indicating a probability that said image data point is part of a lesion"), and
the analysis evaluation unit calculates uncertainty of the classification for each pixel (Mavroeidis paragraph [0062] "The uncertainty data 24 comprises uncertainty data points. Each uncertainty data point corresponds to a respective probability data point (and therefore a respective image data point). Each uncertainty data point is indicative of a level of uncertainty of the indicated probability, e.g. a measure or value representing a (un)certainty that the probability data point is correct. Thus, there are a same number of uncertainty data points as there are probability data points, and therefore of image data point"), and evaluates the determination result on a basis of a distribution of classes on the image (Mavroeidis paragraph [0060] "The probability data 23 comprises probability data points. Each probability data point corresponds to a respective image data point, and is indicative of a probability that the respective image data point forms part of a lesion, such as a tumor or calcium deposit. Thus, there are a same number of probability data points as there are image data points") and a distribution of uncertainty on the image (Mavroeidis paragraph [0062] "The uncertainty data 24 comprises uncertainty data points. Each uncertainty data point corresponds to a respective probability data point (and therefore a respective image data point). Each uncertainty data point is indicative of a level of uncertainty of the indicated probability, e.g. a measure or value representing a (un)certainty that the probability data point is correct. Thus, there are a same number of uncertainty data points as there are probability data points, and therefore of image data point").“
Regarding claim 4, Mavroeidis teaches “The determination evaluation device according to claim 2, wherein the analysis evaluation unit evaluates the determination result of the image as low uncertainty when an average value (Mavroeidis paragraph [0081] "One approach to implement this could be to change the inclusion threshold based on the level of uncertainty for the same image data point or an average uncertainty of the associated predicted lesion") of the uncertainty in the image is smaller than a predetermined threshold (Mavroeidis paragraph [0111-0112] "the boundaries selected for presentation may be those associated with a boundary uncertainty above a first predetermined uncertainty value and a lesion uncertainty below a second predetermined uncertainty value. The first and second predetermined uncertainty values may be the same. The first and second predetermined uncertainty values are preferably in the range of 40-70% of the maximum possible uncertainty value […] Thus, boundaries for lesions that are uncertain may be presented to a user for modification and/or rejection").“
Regarding claim 6, Mavroeidis teaches “The determination evaluation device according to claim 2, further comprising:
an evaluation result display unit that displays a screen based on the evaluation of the determination result (Mavroeidis paragraph [0115] "Thus, for each identified boundary, a graphical annotation may be generated that identifies the location and extend of the identified boundary. This graphical annotation may be designed to overlay the image data (e.g. on a display) so as to indicate the location and/or presence of a boundary of a lesion. The graphical annotations may be contained in boundary data"), wherein
the evaluation result display unit displays an image that is the input data, an image of a distribution of a class on the image that is the determination result (Mavroeidis paragraph [0115] "This graphical annotation may be designed to overlay the image data (e.g. on a display) so as to indicate the location and/or presence of a boundary of a lesion. The graphical annotations may be contained in boundary data"), and an image of a distribution of uncertainty on the image (Mavroeidis paragraph [0062] "The uncertainty data 24 comprises uncertainty data points. Each uncertainty data point corresponds to a respective probability data point (and therefore a respective image data point). Each uncertainty data point is indicative of a level of uncertainty of the indicated probability, e.g. a measure or value representing a (un)certainty that the probability data point is correct. Thus, there are a same number of uncertainty data points as there are probability data points, and therefore of image data points" and paragraph [0138] "The display unit 63 is adapted to receive information about the boundaries identified by the processing system 61 (e.g. boundary information) and display this information, e.g. on a display 63A. The display unit may be adapted to display the medical image data associated with the boundaries, e.g. beneath the illustrated boundaries. This medical image data may be received directly from the image data generator 62" ).“
Regarding claim 7, Mavroeidis teaches “The determination evaluation device according to claim 2, wherein the uncertainty calculation unit sets, as uncertainty, a variance value of a class of each pixel calculated using a Monte Carlo dropout (Mavroeidis paragraph [0132] "The uncertainty data can be calculated using the Manto Carlo dropout approach. In particular, the machine learning model can be trained with dropout layers in the model architecture. During inference of the probability data, we have dropout "turned on", meaning that if the inference is run multiple times, then we will get different probability data (i.e. different predictions). Uncertainty can then be measured as the variance of these predictions. The skilled person would know other methods of calculating uncertainty data").“
Claim 8 recites a method with steps corresponding to the device elements
recited in claim 1. Therefore, the recited steps of this claim are mapped in the same manner as the corresponding elements of device claim 1.
Claim 9 recites a program with steps corresponding to the device elements
recited in claim 1. Therefore, the recited steps of this claim are mapped in the same manner as the corresponding elements of device claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Mavroeidis et al. (US 2022/0180516 A1) in view of Kenemer et al. (US 11,394,732 B1).
Regarding claim 5, Mavroeidis teaches “The determination evaluation device according to claim 4, further comprising:
an evaluation result display unit that displays a screen based on the evaluation of the determination result (Mavroeidis paragraph [0115] "Thus, for each identified boundary, a graphical annotation may be generated that identifies the location and extend of the identified boundary. This graphical annotation may be designed to overlay the image data (e.g. on a display) so as to indicate the location and/or presence of a boundary of a lesion. The graphical annotations may be contained in boundary data"), wherein the evaluation result display unit recommends, for an image evaluated as low uncertainty, (Mavroeidis paragraph [0125] "An initialized machine-learning algorithm is applied to each input data entry to generate predicted output data entries. An error or loss function between the predicted output data entries and corresponding training output data entries is used to modify the machine-learning algorithm. This process can repeated until the error converges, and the predicted output data entries are sufficiently similar ( e.g. ±1%) to the training output data entries. This is commonly known as a supervised learning technique").”
However, Mavroeidis is not relied on to teach “correcting a determination result of a place with high uncertainty of the image”.
Kenemer teaches “correcting a determination result of a place with high uncertainty of the image (Kenemer column 5 lines 51-59 "Display 160 generally represents any type or form of device capable of visually displaying information ( e.g., to a user). In some examples, display 160 may present a graphical user interface. In non-limiting examples, display 160 may present at least a portion of information indicating one or more of input sample 121, security action 122, machine learning model 123, result 124, internal activity 125, activation entropy 126, first threshold 127, combination 128, and/or low-confidence result 129")”.
It would have been obvious to a person having ordinary skill in the art before
effective filing date of the claimed invention of the instant application to combine a device for evaluating classification for boundary area within an image as taught by Mark to include a display that provides user information on the determination results and notification to update the model as taught by Kenemer.
The suggestion/motivation for doing so would have been “In some embodiments, the provided techniques may advanta-geously improve accuracy of classifiers. In examples, the provided techniques may advantageously mitigate data drift with time and/or variation across computing devices and thus may enhance performance of classifiers. In some embodiments, the provided techniques may advantageously be implemented with low overhead”.
Therefore, it would have been obvious to combine the disclosure of Mavroeidis with the Kenemer disclosure to obtain the invention as specified in claim 5 as there is a
reasonable expectation of success and/or because doing so merely combines prior art
elements according to known methods to yield predictable results.
Allowable Subject Matter
Dependent claim 3 is objected to as being dependent upon rejected base claim, but would be allowable if: (i) rewritten in independent form including all the limitations of the base claim and any intervening claims; and (ii) the rejection of these claims under 35 U.S.C. 101, as set forth in this Office Action, is overcome.
The following is a statement of reason for indication of allowable subject matter:
Browning et al. (US 20220270248 A1) discloses a method and system for detecting anatomical landmarks and predicted locations of the landmarks. Uncertainty for the predicted locations of the landmarks is determined by calculating an average full width half maximum from a distribution.
Browning teaches differs from the instant application, because Browning is directors towards spatial uncertainty while the instant application is directed towards uncertainty in classification.
Okawa (WO 2023119664 A1) discloses a method and system for classification of an image by assigning a class to each pixel. A classification score/confidence score for each pixel is determines and then an average of all the pixels classification score is taken to determine whether the confidence in the classification results.
Okawa does not reference any full width half maximum for calculating uncertainty.
Therefore, none of the cited prior art references, alone or in combination provide motivation to teach the ordered combinations of the limitations as recited in claim 3.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASPREET KAUR whose telephone number is (571)272-5534. The examiner can normally be reached Monday - Friday 7:30 am - 4:00 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASPREET KAUR/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights