DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed 02/21/2023 has been considered except where lined through because the listed NPL (“Improved Algorithms for Linear Stochastic Bandits”) is illegible. The Examiner has included a legible copy of the NPL.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, at Step 1, the claim is directed to a an optimization device, which is a statutory category of invention (Machine).
At Step 2A Prong 1, Examiner notes that the claims are directed towards an abstract idea. The claim language has been reproduced below:
a memory configured to store instructions;
and one or more processors configured to execute the instructions to: acquire a reward obtained by executing a certain policy (mathematical relationship);
update a probability distribution of the policy based on the obtained reward (mathematical relationship);
and determine the policy to be executed, based on the updated probability distribution (mathematical relationship), wherein the probability distribution is updated by using a weighted sum of the probability distributions updated in a past as a constraint (mathematical calculation).
At Step 2A Prong 2, the additional elements are bolded above. The additional elements do not integrate the abstract ideas into a practical application because the computer elements, which are recited at a high level of generality, provide conventional computer functions that do not impose any meaningful limits on practicing the abstract ideas. See MPEP 2106.05(f). The limitations “memory” and “one or more processors” are merely generic computer components recited at a high level of generality. The limitation “store instructions” is an insignificant extra-solution activity of data storage. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application, and the claim is directed to the judicial exception.
At Step 2B, the additional elements do not, alone or in combination, amount to significantly more than the recited judicial exception. As set forth in step 2A prong 2 analysis, the functions of “store instructions” is recognized by the courts as well-understood routine and conventional. See MPEP 2106.05(d)(II). Furthermore, the “memory” and “one or more processors” are the equivalent of adding the words “apply it” to the judicial exception and are mere instructions to implement the abstract idea on a computer. Even when considered in combination, these additional elements represent mere instructions to apply an exception and insignificant extra-solution activity, which do not provide an inventive concept. The claim is not eligible.
Regarding claims 2-6, the claims merely recite functions for updating the probability distribution that further mathematically limit the mathematical concepts, or provide additional mathematical functions, of claim 1. They do not include additional elements that would require further analysis under steps 2A prong 2 and step 2B.
Regarding claim 7, the claim is directed to a method that would be practiced by the device of claim 1. All steps performed by the method of claim 7 are executed by the device in claim 1 as configured. The analysis of claim 1 applies equally to claim 7.
Regarding claim 8 the claim is directed to a non-transitory computer-readable recording medium that would be practiced by the device of claim 1. All steps performed by the non-transitory computer-readable recording medium of claim 8 are executed by the device in claim 1 as configured. The analysis of claim 1 applies equally to claim 8.
Allowable Subject Matter
Claims 1-8 would be allowable if rewritten to overcome the rejections under 35 USC 101 set forth in this Office Action.
The following is a statement of reasons for the indication of allowable subject matter:
As to claim 1, the prior art of record does not teach or suggest a combination as claimed including: one or more processors configured to execute the instructions to: acquire a reward obtained by executing a certain policy; update a probability distribution of the policy based on the obtained reward; and determine the policy to be executed, based on the updated probability distribution wherein the probability distribution is updated by using a weighted sum of the probability distributions updated in a past as a constraint.
Ghosh et al. (US 20210089959 A1, hereinafter “Ghosh”) discloses a contextual bandit algorithm that may update the probability distribution for each collection when receiving feedback ([0022]) and selects a collection it predicts will yield the highest probability of success ([0021]) with the highest reward ([0027]). Ghosh does not suggest selecting a collection based on the updated probability distribution nor the updating using a weighted sum of past probability distributions. Therefore, Ghosh does not teach or suggest a combination as claimed including the limitations identified above.
Bubeck et al. (The Best of Both Worlds: Stochastic and Adversarial Bandits, hereinafter “Bubeck”) discloses a stochastic and adversarial optimal bandit algorithm that alternates between the two arms until one appears significantly better, then focuses on the better arm while occasionally resampling the other arm (Section 3.1). Bubeck does not suggest updating the probability distribution of each arm nor determining the arm based on the updated probability distribution. Therefore, Bubeck does not teach or suggest a combination as claimed including the limitations identified above.
Zimmert et al. (An Optimal Algorithm for Stochastic and Adversarial Bandits, hereinafter “Zimmert”) discloses a bandit algorithm that optimizes pseudo-regret in adversarial and stochastic multi-armed bandits without prior knowledge using online mirror descent (abstract). Zimmert does not suggest updating the probability distributions using a weighted sum of past probability distributions. Therefore, Zimmert does not teach or suggest a combination as claimed including the limitations identified above.
Cheung et al. (Learning to Optimize under Non-Stationarity, hereinafter “Cheung”) discloses a stochastic bandit algorithm optimized for non-stationarity (Section 1) which designs a policy to maximize the cumulative reward (Section 2.2). Chueng does not suggest updating the probability distribution of each policy nor determining the policy based on the updated probability distribution, Therefore, Cheung does not teach or suggest a combination as claimed including the limitations identified above.
St-Pierre et al. (Differential Evolution algorithm applied to non-stationary bandit, hereinafter “St-Pierre”) discloses obtaining rewards of policies with unknown distributions (Section II.A). St-Pierre does not suggest updating the probability distribution of each policy nor determining the policy based on the updated probability distribution. Therefore, St-Pierre does not teach or suggest a combination as claimed including the limitations identified above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHAT N LE whose telephone number is (571)272-0546. The examiner can normally be reached Monday-Friday 8:30AM-5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew T Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.N.L./
Phat LeExaminer, Art Unit 2182 (571) 272-0546
/ANDREW CALDWELL/Supervisory Patent Examiner, Art Unit 2182
Prosecution Insights