Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Objections Claim 10 is objected to because of the following informalities: The word identifying in identifying a target policy should read identify . Claim 14 is objected to because of the following informalities: The a in And is inappropriately capitalized. These appear to b e typographical errors. Appropriate correction is required. Claim Interpretation The term c omputer readable storage medium is interpreted, as defined in [0016] of the specification, not to include transitory signals per se. The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are a machine learning system for implementing a method, the system configured to: collect, identify[ ], calculate, identify, calculate, and calculate . Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 112 The follo wing is a quotation of 35 U.S.C. 112(b): (b ) CONCLUSION.— The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the appl icant regards as his invention. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 1 recites the variables s i , a i , r i , s’ i , i , n , s , and a , none of which are defined in the claim and are thus indefinite. It is also unclear whether the dataset D and the set Ɗ refer to the same set or not, nor whether the functions σ (s,a) and the value σ refer to the same entity or not, nor whether the functions w (s,a) and the value w refer to the same entity or not. Claim 1 further Claim 4 recites the limitation the importance sampling . There is insufficient antecedent basis for this limitation in the claim, and will be interpreted as if it had read the importance sampling. Claims 10 and 19 have identical limitations that also are indefinite for the same reasons. Claim 2 recites the abbreviation MWL without any explanation of the acronym, rendering the claim indefinite. Claims 11 and 20 have identical limitations that are indefinite for the same reason. Claim 3 also recites additional variables β , R , and T which are undefined and thus indefinite. Claim 12 has identical limitations that are indefinite for the same reasons. Claim 4 recites the limitations the importance sampling estimate and the error estimate . There is insufficient antecedent basis for th ese limitation s in the claim , and will be interpreted as if it had read an importance sampling estimate and an error estimate . Claim 1 3 has identical limitations that are indefinite for the same reasons. Claim 6 recites the limitations the output of φ . There is insufficient antecedent basis for this limitation in the claim, and will be interpreted as if it had read an output of φ . Claim 1 5 has identical limitations that are indefinite for the same reasons. Regarding claim 7 , the phrase " e.g. " renders the claim indefinite because it is unclear whether the limitation(s) following the phrase are part of the claimed invention. See MPEP § 2173.05(d). Claim 16 has an identical limitation and is indefinite for the same reason. Claim 8 recites the phrase the support function has about the same support , where the term about is a relative term of degree. The term about is not defined by the claims nor the specification and a person having ordinary skill in the art would not understand the scope of the claim, and thus the claim is indefinite. Claim 8 further recites the limitation the probability density function , which lacks proper antecedent basis in the claims. Claim 17 has an identical limitation and is indefinite for the same reason. Claim 10 recites the limitations the system after previously reciting a computing system and a machine learning system . It is thus indefinite to which system the recited system refers. For the purpose of examination, the claim will be interpreted to mean the machine learning system . Claim limitation the machine learning system i n Claim 10 i nvokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. No structure, such as processor, is ever explicitly disclosed in the specification as the corresponding structure for the machine learning system . Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. Applicant may: (a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph; (b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or (c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)). If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either: (a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or (b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181. Dependent claims are rejected for inheriting the indefiniteness of a parent claim. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 1 recites a method , thus a process, one of the four statutory categories of patentable subject matter. However, the claim further recites collecting a dataset D (interpreted in light of Claim 3 as sampling from a probability distribution, thus a mathematical calculation); identifying a target policy (a mental process); calculating a support function estimate (a mathematical calculation) ; identifying a discriminator set (a mental process); calculating an importance weight estimate (a mathematical calculation); and calculating a minimax interval estimate (a mathematical calculation). Thus, the claim recites an abstract idea of calculating a minimax interval estimate from data. The claim does not include any additional elements at all to the abstract idea, and thus cannot include any additional elements which could integrate the abstract idea into a practical application nor provide significantly more than the abstract idea itself. Therefore, the claim is subject matter ineligible. Dependent Claims 2-9 exclusively recite mathematical calculations, i.e. additional steps in the abstract idea, and thus cannot recite any additional elements which could integrate the abstract idea into a practical application nor provide significantly more than the abstract idea itself. Claims 10-18 recite a computing system comprising a machine learning system for implementing precisely the method of Claims 1-9 . As the means-plus-function interpretation of the claims requires interpretation of the machine learning system as including structure such as a processor (even though such structure is not explicitly disclosed), the claims recite an article of manufacture, one of the four statutory categories of subject matter. However, as the claims recite merely generic computer components to perform the identical abstract idea of Claims 1-9, respectively (which can neither integrate the abstract idea into a practical application nor provide significantly more than the abstract idea itself, see MPEP 2106.05(f)(2)), Claims 10-18 are rejected for reasons set forth in the rejections of Claims 1-9, respectively. Similarly, Claims 19 and 20 recite a computer program product comprising a computer readable storage medium (interpreted in light of the special definition in the specification as a non-transitory computer readable medium) to perform the methods of Claims 1 and 4, respectively, and are similarly rejected, in view of MPEP 2106.05(f)(2), as reciting an abstract idea, merely performed on generic computer equipment. Claim Rejections - 35 USC § 102 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis ( i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale , or otherwise available to the public before the effective filing date of the claimed invention. Claim s 1 , 3 , 4 ; 10 , 12 , 13 ; 19 and 20 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Dai et al., “ CoinDICE : Off-Policy Confidence Interval Estimation.” 3838575 1407160 Regarding Claim 1 , Dai teaches a method of performing an Offline Policy Evaluation (OPE) based on importance sampling (Dai, Title, “Off-Policy Confidence Interval Estimation” & Abstract, “the goal is to estimate a confidence interval on a target policy’s value, given only access to a static experience dataset collected by unknown behavior policies” i.e. offline ) , comprising: collecting a dataset D, wherein D is given by 2438400 645160 (Dai, pg. 2, 3 rd paragraph, “We are interest ed in estimating the policy value and its confidence interval in the behavior agnostic off-policy setting, where interaction with the environment is limited to a static dataset of experience ” ), identifying a target policy π (Dai, Abstract, “the goal is to estimate a confidence interval on a target policy ’s value, given only access to a static experience dataset collected by unknown behavior policies”) ; calculating a support function estimate σ (s,a) where the support function estimate σ s,a is responsive to the dataset D (Dai, pg. 4, 2 nd -to-last paragraph, “ ”) ; identifying a discriminator set F (Dai, pg. 4, Eq. 8, “ “) ; calculating an importance weight estimate w s,a responsive to the dataset D, the target policy π, the support function estimate σ , and the discriminator set F (Dai, pg. 5, Eq, 12, where w l and w u are importance weight estimates, i.e. “optimal weights for the upper and lower confidence bounds”) and calculating a minimax interval estimate C responsive to the dataset D and the importance weight estimate w (Dai, pg. 5, Eq. (13): ”). Regarding Claim 3 , Dai teaches the method of Claim 1 (and thus the rejection o Claim 1 is incorporated). Dai further teaches wherein the dataset D is sampled from a distribution given by (Dai, pg. 2, Section 2, “ ” where these are equivalent mathematical descriptions of the same distribution ). Regarding Claim 4 , Dai teaches the method of Claim 1 (and thus the rejection o f Claim 1 is incorporated). Dai further teaches wherein the minimax interval estimate C is given by (Dai, because any interval is given by its midpoint and half-width, Eq. (13) has this form ). Claims 10 , 12 , and 13 recite a computing system, comprising a machine learning system for implementing precisely the methods of Claims 1, 3, and 4, respectively. As Dai implements their method on a computer (Dai, pg. 1, Footnote 1, “Open-source code for CoinDICE is available”), Claims 10, 12, and 13 are rejected for reasons set forth in the rejections of Claims 1, 3, and 4, respectively. Similarly, Claims 19 and 20 recite a computer program product comprising a computer readable storage medium having program instructions to perform the methods of Claims 1 and 4, respectively, on a processor, and are thus also rejected for reasons set forth in the rejections of Claims 1 and 4, respectively. Claim Rejections - 35 USC § 103 The follo wing is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness . This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. Claim s 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Dai et al., “ CoinDICE : Off-Policy Confidence Interval Estimation,” in view of Feng, “Offline Reinforcement Learning via an Optimization Lens.” Regarding Claim 6 , Dai teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Dai does not teach where the discriminator set F is given by but Feng , also in the field of off-policy reinforcement learning interval estimation, teaches this limitation (Feng, pg. 86, 3 rd -to-last paragraph, “ “). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the basis of the kernel, as does Feng, in the policy interval optimization of Dai. The motivation to do so is “to speed up the computation” (Feng, pg. 84). Claim 1 5 recites a computing system, comprising a machine learning system for implementing precisely the method of Claim 6. As Dai implements their method on a computer (Dai, pg. 1, Footnote 1, “Open-source code for CoinDICE is available”), Claim 15 is rejected for reasons set forth in the rejection of Claim 6. Claim s 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Dai et al., “ CoinDICE : Off-Policy Confidence Interval Estimation,” in view of Kanamori et al., “A Least-squares Approach to Direct Importance Estimation,” as provided by the applicant in the Information Disclosure Statement dated 6/27/2023. Regarding Claim 9 , Dai teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Dai does not teach where the support function estimate σ s,a is computed using an unconstrained Least-Square Importance Fitting ( uLSIF ) algorithm , but Kanamori, also in the field of importance estimation, teaches this limitation (Kanamori, pg. 1403, Section 3.1, “Unconstrained Least-squares formulation” for estimating the importance ). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Kanamori’s uLSIF algorithm to computer the support function estimate of Dai. The motivation to do so is that it is “computably efficient and reliable” (Kanamori, pg. 1403, Section 3) way to estimate the density of the support of the sampled data. Claim 1 8 recites a computing system, comprising a machine learning system for implementing precisely the method of Claim 9. As Dai implements their method on a computer (Dai, pg. 1, Footnote 1, “Open-source code for CoinDICE is available”), Claim 18 is rejected for reasons set forth in the rejection of Claim 9. Conclusion Claims 2 and 5 (and relatedly, their dependents 7 and 8 as well as system Claims 11 and 14-16) have been searched, but no prior art which teaches the recited mathematical expression for the error estimate has been uncovered. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Xie et al., “Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling” teaches an importance sampling method for estimating the confidence interval of a target policy based on off-policy sampling. Any inquiry concerning this communication or earlier communications from the examiner should be directed to FILLIN "Examiner name" \* MERGEFORMAT BRIAN M SMITH whose telephone number is FILLIN "Phone number" \* MERGEFORMAT (469)295-9104 . The examiner can normally be reached FILLIN "Work Schedule?" \* MERGEFORMAT Monday - Friday, 8:00am - 4pm Pacific . Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, FILLIN "SPE Name?" \* MERGEFORMAT Kakali Chaki can be reached at FILLIN "SPE Phone?" \* MERGEFORMAT (571) 272-3719 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /BRIAN M SMITH/ Primary Examiner, Art Unit 2122