Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to because the legend of Fig. 12 identifies the plotted curve G1 as “THIRD EXAMPLE EMBODIMENT,” whereas the specification states that Fig. 12 illustrates an Example using the information processing apparatus 1C of the fourth example embodiment (¶[0013], ¶[0150]); and the lead lines for reference characters G2 and G3 terminate on what appears to be the same plotted line, so that the curves for Related Technique A and Related Technique B cannot be distinguished. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “Information Processing Apparatus, Method, System, and Storage Medium Using a Plurality of Independently Learning Predictors to Determine an Option to Be Selected”.
The disclosure is objected to because of the following informalities: (a) ¶[0077] refers to “the information processing method S10,” whereas the method shown in Fig. 7 and described in ¶[0070] is designated S1B; (b) ¶[0109] recites “the storage unit 150B” in the description of information processing apparatus 1C, whereas the storage unit of apparatus 1C is 150C (¶[0103], ¶[0106]; Fig. 9); (c) ¶[0131] states that method S1C includes steps S11C through S18C, and Fig. 10 shows step S17C as “RECEIVE PROMOTION RESULT” and step S18C as “ACCUMULATE TRAINING DATA IN ASSOCIATION WITH PREDICTOR USED IN SELECTION,” whereas ¶[0137]–¶[0138] describe no receiving step and describe accumulation as step S17C, and no step S18C is described; (d) the determination-condition boundary is described inconsistently: ¶[0067] and ¶[0085] describe the not-satisfied case as a prediction error “greater than” the first threshold and, respectively, “equal to or greater than” the first threshold, and ¶[0068] recites “equal to or greater than a predetermined value” while ¶[0083] recites “greater than a predetermined value” for the same extraction.
Appropriate correction is required.
Claim Objections
Claims 2, 3, 7, and 9 are objected to because of the following informalities: in claim 2, part (iii) the clause “a prediction error of each of the one or more options which have been extracted is predicted, with use of another predictor, as an option group to be dealt with by the another predictor” is grammatically unclear; in claim 3, “each of prediction errors” should be “each of the prediction errors”; in claim 7, “the option determined had been selected” should be “the option determined has been selected,” and “in order to determine an option to be selected from among the plurality of options, any of a plurality of predictors being used” is grammatically awkward and should be rewritten as a step of the method (e.g., “determining the option with use of any of a plurality of predictors”); in claim 9, “an information processing apparatus recited in claim 1” should be “the information processing apparatus recited in claim 1.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 2-4, 7 and 9 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites "the predictor" in limitations (i), (ii) and (iii). There is insufficient antecedent basis for this limitation in the claim. Claim 1 recites "a plurality of predictors," so it is unclear which predictor "the predictor" refers to, or whether the same predictor is meant in each limitation. For purposes of examination, "the predictor" is interpreted under BRI as the one predictor, from among the plurality of predictors, that is currently used in the determination process.
Claim 2 recites "with use of another predictor, as an option group to be dealt with by the another predictor" in limitation (iii). The phrase is grammatically unclear, and it cannot be determined what constitutes "an option group to be dealt with by the another predictor." "The another predictor" also lacks proper antecedent basis. For purposes of examination, limitation (iii) is interpreted under BRI to mean that the extracted options form the option group of a next predictor, which then predicts a prediction error for each extracted option.
Claim 2 recites "an option group" in limitations (i) and (ii) and "the option group" in limitation (iii). It is unclear whether the option groups are the same group. For purposes of examination, the option groups of (i), (ii) and (iii) are interpreted under BRI as the same option group dealt with by the current predictor.
Claim 3 recites "a condition that includes a condition in which each of prediction errors is equal to or lower than a threshold." The phrase is circular, and it is unclear which prediction errors are meant by "each of prediction errors." Claim 3 also inherits the indefiniteness of claim 2. For purposes of examination, the condition is interpreted under BRI as requiring that every predicted prediction error be at or below a threshold.
Claim 7 recites "in order to determine an option to be selected from among the plurality of options, any of a plurality of predictors being used, and the plurality of predictors independently learning a relation..." It is unclear whether the passive "being used" and "learning" are required method steps or only an intended result. The claim also re-recites "an option to be selected from among the plurality of options," so it is unclear whether this is the option from the determining step. For purposes of examination, the clause is interpreted under BRI as requiring that the determining step use one of a plurality of predictors that independently learn the relation from the training data. The determined option is treated as the same option recited in the original determining step.
Claim 9 recites "an acquisition process" performed by the server, but claim 1, from which it depends, already recites "an acquisition process." It is unclear whether these are the same process or two distinct processes. For purposes of examination, the server's acquisition process is interpreted under BRI as a separate process, performed by the server, that acquires the relevant information and sends it on to the apparatus.
Claim 9 recites "the relevant information" in the singular, but claim 1 recites "pieces of relevant information." There is insufficient antecedent basis for this limitation in the claim. For purposes of examination, "the relevant information" is interpreted under BRI as the pieces of relevant information of claim 1.
Claim 9 recites "information received from the information processing apparatus." No claim recites that the apparatus transmits information, so it is unclear what information is received and how the selected option relates to the "option determined" in claim 1. Claim 9 also recites "a gain" and "an observation value" after claim 1 already recites them, and it is unclear whether they are the same. For purposes of examination, the received information is interpreted under BRI as information identifying the option determined by the apparatus, and the gain and observation value are interpreted as the same as in claim 1.
Claim 4 is rejected as depending from rejected claim 2 and inheriting its indefiniteness.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis below follows the 2019 Revised Patent Subject Matter Eligibility Guidance as incorporated in MPEP 2106, the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128 (July 17, 2024), and Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB Sept. 26, 2025) (Appeals Review Panel Decision). Each claim has been given its broadest reasonable interpretation in light of the specification (MPEP 2106, subsection II; MPEP 2111).
CLAIM 1
Step 1: Claim 1 recites “An information processing apparatus, comprising at least one processor” that carries out an acquisition process, a determination process, and an accumulation process. An apparatus comprising a processor falls within the statutory category of a machine. See MPEP 2106.03. Accordingly, claim 1 satisfies Step 1 of the eligibility analysis.
Step 2A, Prong 1: Claim 1 recites an abstract idea, namely a mental process. Specifically, claim 1 recites “a determination process of determining an option to be selected from among the plurality of options” and specifies that, “in the determination process, an option to be selected from among the plurality of options being determined.” Under the broadest reasonable interpretation in light of the specification, this limitation sets forth choosing one option from a set of candidate options on the basis of information associated with the options and the gains observed when options were previously selected. The specification describes this as the decision made in a “bandit” problem, in which one of a plurality of options is repeatedly selected, the resulting gain is observed, and each new option is determined “by referring to observation values of gains obtained in previous trials” (¶[0002]; see also ¶[0015], [0019]), and its principal example is deciding which user should be sent a product promotion based on information about the users and on whether earlier recipients purchased the product (¶[0092]-[0094]). Choosing among candidate options in this way is an evaluation and judgment that can practically be performed in the human mind or with pen and paper; for example, a salesperson can review notes on each prospective customer, recall how similar customers responded to earlier offers, and decide which customer to approach next. Analyzing information by steps that can practically be performed in the mind is a mental process (Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1353-54 (Fed. Cir. 2016)), and it remains one when the claim recites that a computer performs it (MPEP 2106.04(a)(2), subsection III(C); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1335 (Fed. Cir. 2015)). See MPEP 2106.04(a)(2), subsection III.
The recitation that the option is determined “with use of any of a plurality of predictors” does not take the determination outside the mental processes grouping; the determination of which option to select remains the mental evaluation, and the predictors used as a tool to make it are evaluated below as additional elements, consistent with the USPTO's analysis of the limitation “detecting one or more anomalies in a data set using the trained ANN” in claim 2 of Example 47 of the July 2024 Subject Matter Eligibility Examples.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application.
The additional elements are: (a) the “information processing apparatus” and the “at least one processor” that carries out the claimed processes; (b) “an acquisition process of acquiring pieces of relevant information which are respectively associated with a plurality of options”; (c) “an accumulation process of accumulating, as training data, an observation value of a gain which has been obtained when the option determined in the determination process had been selected and relevant information of the option in a storage apparatus”; and (d) determining the option “with use of any of a plurality of predictors,” with “the plurality of predictors independently learning a relation between the relevant information and the observation value with reference to the training data.”
The additional elements, considered individually and as an ordered combination, do not integrate the judicial exception into a practical application. See MPEP 2106.04(d).
Under the improvements consideration (MPEP 2106.04(d)(1); MPEP 2106.05(a)), the specification has been evaluated, as directed by Ex Parte Desjardins, to determine whether it describes an improvement in the functioning of a computer, or an improvement to another technology or technical field, and whether claim 1 reflects that improvement. The specification identifies the problem as the contextual bandit algorithm of Non-patent Literature 1 having “room for improvement in terms of an increase in accumulation of observation values,” because when “a learned relation between an observation value and a context deviates from an original relation thereof, an option different from an optimal option can be selected” (¶[0005]; see ¶[0048]-[0049]), and it states that the object and effect of the invention is to “further increase accumulation of observation values” (¶[0006], [0012]). For the configuration recited in claim 1, the specification states only that, because the predictors learn independently, “it is possible to use . . . a predictor which has more appropriately learned a relation between an observation value and relevant information,” which “makes it possible to determine an option that further increases accumulation of gains” (¶[0026]; see also ¶[0164]). That asserted benefit is a better selection outcome: choosing options that yield more of the gain being measured, which in the specification's example means more purchases by the users who receive promotions (¶[0094], [0154]). Making better selections improves the abstract selection decision itself; it does not improve the functioning of the processor, the storage apparatus, or any other technology, and an improvement in the abstract idea itself is not an improvement in technology (MPEP 2106.05(a), discussing Trading Technologies Int'l, Inc. v. IBG LLC, 921 F.3d 1084, 1093-94 (Fed. Cir. 2019); see also SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163 (Fed. Cir. 2018) (an advance that “lies entirely in the realm of abstract ideas” is ineligible)). The specification does not describe any reduction in processing, memory, or storage requirements, any reduction in system complexity, or any other change in how the computer operates, and the same benefit would be obtained by anyone who applied the same selection rule. This differs from Desjardins, where the specification described improvements in how the machine learning model itself operates (learning new tasks while protecting knowledge of earlier tasks to overcome “catastrophic forgetting,” together with reduced storage and reduced system complexity) that were not subsumed in the recited mathematical calculation. Moreover, the statement in ¶[0026] of specification is a conclusory assertion as applied to claim 1: the specification does not explain how merely determining an option “with use of any of” several independently learning predictors yields the asserted benefit.
Even if the specification were found to describe a technological improvement, claim 1 does not reflect it. The specification attributes its demonstrated results to a particular mechanism: using the predictors one after another in a fixed order, predicting a prediction error for each option, comparing the prediction errors with thresholds, passing a narrowed group of options from one predictor to the next, and, in the fourth example embodiment, using thresholds that depend on the dimensionality of the feature information (¶[0059]-[0069], [0086]-[0089], [0113]-[0129], [0149]-[0154]). None of these steps is recited in claim 1, which requires only that an option be determined “with use of any of a plurality of predictors” that independently learn a relation from the training data.
Under its broadest reasonable interpretation, claim 1 encompasses determining an option with any one of several conventionally trained predictors, even one chosen arbitrarily, which the specification does not describe as producing any improvement. A claim that lacks limitations addressing the asserted improvement does not reflect that improvement (Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1316 (Fed. Cir. 2016); MPEP 2106.05(a)). Viewed as a whole, and without oversimplifying its limitations, claim 1 recites the idea of a solution (using several independently trained predictors to decide which option to select) rather than a particular way of achieving a technological result. See MPEP 2106.05(f)(1).
Considering each additional element in turn: (a) The “information processing apparatus” and “at least one processor” are recited at a high level of generality and serve only to carry out the abstract determination and the related handling of data. The specification confirms that the apparatus may be any computer having “at least one processor C1 and at least one memory C2,” with the processor being, for example, a CPU, GPU, DSP, MPU, FPU, PPU, or microcontroller (spec ¶[0157]-[0158]). Using generic computer components as tools to perform an abstract idea is no more than mere instructions to apply the exception on a computer (MPEP 2106.05(f); Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 223-26 (2014)). (b) The “acquisition process” gathers the relevant information that serves as input to the determination; it is mere data gathering and insignificant pre-solution activity (MPEP 2106.05(g); OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015); CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1375 (Fed. Cir. 2011)). (c) The “accumulation process” records the observed gain of the selected option, together with that option's relevant information, as training data in a “storage apparatus.” Every use of a selection rule that is learned from past results requires gathering and storing those results, so this limitation is insignificant extra-solution activity in the form of necessary data gathering and storage (MPEP 2106.05(g), consideration (3); see OIP Techs., 788 F.3d at 1363 (gathering statistics about how customers responded to offers); Ultramercial, Inc. v. Hulu, LLC, 772 F.3d 709, 715 (Fed. Cir. 2014) (consulting and updating an activity log)). The storage apparatus itself is generic memory used for its ordinary purpose (spec ¶[0020], [0158]; MPEP 2106.05(f)). (d) Determining the option “with use of any of a plurality of predictors,” with the predictors “independently learning a relation between the relevant information and the observation value with reference to the training data,” recites machine learning only at a high level of generality. The claim does not specify the type of predictor, how a relation is learned, or how a predictor's output is used to determine the option, and “independently” requires only that the predictors learn separately, for example from different training data sets (¶[0019]). The limitation recites the result of learning a relation and using it, with no description of a mechanism for achieving a technological result. It invokes generic machine learning as a tool to perform the selection decision, which is mere instructions to apply the exception (MPEP 2106.05(f)(1)-(2)), and it generally links the exception to the field of machine learning (MPEP 2106.05(h)). Training on data from previous trials and applying the learned relation to new data are inherent in any use of machine learning, and such incidental features of generic machine learning do not integrate an abstract idea (Recentive Analytics, Inc. v. Fox Corp., 134 F.4th 1205 (Fed. Cir. 2025)).
As an ordered combination, the additional elements are a generic computer that gathers data, applies generic learned predictors to make the abstract selection, and stores the result for further learning, i.e., the abstract idea carried out on a computer with conventional data flow. The combination does not improve the computer or any other technology, does not use a particular machine (MPEP 2106.05(b)), does not transform any article (MPEP 2106.05(c)), and does not otherwise apply the exception in a meaningful way beyond linking it to a computing environment (MPEP 2106.05(e)). Accordingly, claim 1 does not satisfy Step 2A, Prong 2 and is directed to the abstract idea.
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception. As explained above, (a) the generic apparatus and processor and (d) the generically recited plurality of learning predictors are no more than mere instructions to apply the exception with generic computing and generic machine learning; those conclusions carry over to Step 2B, and such elements cannot provide an inventive concept (MPEP 2106.05(f); Alice, 573 U.S. at 223-26). The generic nature of these elements is also evidenced by the specification (MPEP 2106.07(a), subsection III(A)). The specification describes the hardware only as ordinary processors and memories (¶[0157]-[0158]). It describes the learning only as “machine learning using the training data group” that constructs a prediction function and a prediction error function (¶[0064]), with the relation being, “for example, a linear relation” (¶[0063]), and it describes “learning independently” as using training data sets that differ from each other (¶[0019]). That level of description indicates that the elements are sufficiently well-known that no particulars needed to be described. The specification further acknowledges that the contextual bandit algorithm of Non-patent Literature 1 already “learns, in each of trials, a relation between an observation value and a context using, as training data, observation values and contexts which have been obtained in previous trials” and “predicts a gain of each option using the learned relation to determine an option to be selected” (¶[0003]), and it identifies Greedy, LinUCB, and LinTS as “publicly known bandit algorithms” (¶[0150]). Elements (b) and (c), found above to be insignificant extra-solution activity, have been re-evaluated at Step 2B. Claimed at this level of generality, they are well-understood, routine, conventional computer functions: receiving data (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355 (Fed. Cir. 2014); OIP Techs., 788 F.3d at 1363); storing and retrieving information in memory (Versata, 793 F.3d at 1334; OIP Techs., 788 F.3d at 1363); and electronic recordkeeping (Alice, 573 U.S. at 225; Ultramercial, 772 F.3d at 716). See MPEP 2106.05(d), subsection II. The specification likewise describes the acquisition only generically, as acquiring the information “via an input apparatus” or “from another apparatus . . . that is communicably connected” (¶[0018]), and states that the storage apparatus may be included in or connected to the apparatus (¶[0020]). As an ordered combination, the additional elements add nothing beyond what they provide individually: a generic computer receives data, uses generic learned predictors to make the abstract determination, and stores the result for further learning. That is the conventional arrangement of any computerized learning-based decision process, and it provides no inventive concept. Accordingly, claim 1 does not satisfy Step 2B and is rejected under 35 U.S.C. 101.
CLAIM 2
Step 1: Claim 2 depends from claim 1 and recites an information processing apparatus, which falls within the statutory category of a machine. See MPEP 2106.03.
Step 2A, Prong 1: Claim 2 additionally recites, in the determination process, that “(i) a prediction error is predicted . . . for each of one or more options included in an option group to be dealt with by the predictor, the prediction error being involved in a prediction value of the gain”; that “(ii) in a case where a determination condition pertaining to prediction errors is not satisfied, an option to be selected from among options in an option group dealt with by the predictor is determined”; and that “(iii) in a case where the determination condition is satisfied, one or more options are extracted from the option group dealt with by the predictor, and a prediction error of each of the one or more options which have been extracted is predicted . . . as an option group to be dealt with by the another predictor.” Predicting the prediction error involved in a predicted value of the gain is a mathematical calculation. A prediction error is a numerical, statistical measure of the uncertainty of a predicted value, and under the broadest reasonable interpretation in light of the specification it is calculated by applying a prediction error function to the relevant information of each option (¶[0061], [0064]-[0065]), for example by expression (6) (¶[0121]). The limitation thus describes in words a specific calculation (determining the error of a predicted value) rather than merely involving mathematics, and it falls within the mathematical concepts grouping. See MPEP 2106.04(a)(2), subsection I(C). Steps (ii) and (iii) then test whether the calculated prediction errors satisfy a condition: if not, an option is chosen from the current group; if so, the group is narrowed to a subset of options that are evaluated further. Comparing calculated values against a condition, choosing an option, and shortlisting candidates are evaluations and judgments that can practically be performed in the human mind or with pen and paper for a group of candidate options, and they are mental processes. See MPEP 2106.04(a)(2), subsection III. The recitations that the prediction errors are predicted “with use of any of the plurality of predictors” and “with use of another predictor” refer to the predictors already recited in claim 1, which remain additional elements. No new additional elements are introduced.
Step 2A, Prong 2 and Step 2B: No new additional elements are introduced. Therefore, the analysis from parent claim is maintained.
CLAIM 3
Step 1: Claim 3 depends from claim 2 and recites an information processing apparatus, which falls within the statutory category of a machine. See MPEP 2106.03.
Step 2A, Prong 1: Claim 3 additionally recites that “a condition that includes a condition in which each of prediction errors is equal to or lower than a threshold is used as the determination condition.” Testing whether each calculated prediction error is equal to or lower than a threshold is a mathematical relationship (an inequality between numerical values) (MPEP 2106.04(a)(2), subsection I(A)), and it is also a comparison of values against a threshold that can practically be performed mentally or with pen and paper, i.e., a mental process (MPEP 2106.04(a)(2), subsection III).
Step 2A, Prong 2 and Step 2B: No new additional elements are introduced. Therefore, the analysis from parent claim is maintained.
CLAIM 4
Step 1: Claim 4 depends from claim 2 and recites an information processing apparatus, which falls within the statutory category of a machine. See MPEP 2106.03.
Step 2A, Prong 1: Claim 4 additionally recites that “a condition based on dimensionality of the relevant information is used as the determination condition.” The dimensionality of the relevant information is the dimension of the vector space needed to represent that information as a vector, e.g., the number of numerical values that describe an option (spec ¶[0107]). Because the determination condition of claim 2 is a condition “pertaining to prediction errors,” which are numerical values, claim 4 requires a test of the prediction errors whose criterion depends on another number, the dimensionality. That is a mathematical relationship between variables, expressed in words (MPEP 2106.04(a)(2), subsection I(A)). The specification's thresholds computed from the dimensionality d, e.g., expression (2), $\alpha \sqrt{d/T}$, together with expressions (3) and (4) (spec ¶[0113]-[0118]), illustrate that relationship; they are not read into the claim. The limitation therefore falls within the mathematical concepts grouping (MPEP 2106.04(a)(2), subsection I). Evaluating whether the prediction errors of a group of candidate options satisfy a threshold that depends on how many values describe each option can also practically be performed mentally or with pen and paper, and it is a mental process (MPEP 2106.04(a)(2), subsection III).
Step 2A, Prong 2 and Step 2B: No new additional elements are introduced. Therefore, the analysis from parent claim is maintained.
CLAIM 5
Step 1: Claim 5 depends from claim 1 and recites an information processing apparatus, which falls within the statutory category of a machine. See MPEP 2106.03.
Step 2A, Prong 1: Claim 5 incorporates all the limitations of claim 1 and therefore recites the same abstract idea identified in claim 1. Claim 5 additionally recites that the training data is accumulated in the storage apparatus “while a piece of training data is associated with a predictor which has been used in determination of an option in the determination process,” and that “each of the plurality of predictors learns the relation with reference to a piece of training data associated with that predictor from among the training data accumulated in the storage apparatus.” These recitations do not add a further judicial exception; they narrow two additional elements of claim 1. The association of each piece of training data with the predictor used in the determination is recited as the manner in which the training data is accumulated “in the storage apparatus,” and therefore narrows the accumulation process. The learning “with reference to a piece of training data associated with that predictor” narrows the learning by the predictors. The additional elements of claim 5 are therefore those of claim 1, with the training data stored in association with the predictor used and each predictor's learning limited to the training data associated with it.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application. For the same reasons as discussed in the Step 2A, Prong 2 analysis of claim 1, the generic apparatus and processor, the acquisition process, the storage apparatus, and the accumulation of training data do not integrate the judicial exception into a practical application. Storing each piece of training data in association with the predictor used does not change this conclusion. The claim requires only that a stored record be associated with a predictor, for example by an identifier; it recites no particular data structure and no change in how the storage apparatus stores or retrieves data. Organizing stored information in this way improves, at most, the information stored, not the functioning of the storage apparatus or the computer (BSG Tech LLC v. Buyseasons, Inc., 899 F.3d 1281, 1287-88 (Fed. Cir. 2018) (“an improvement to the information stored by a database is not equivalent to an improvement in the database's functionality”); MPEP 2106.05(a); contrast Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1339 (Fed. Cir. 2016) (self-referential table for a computer database directed to an improvement in computer capabilities)). The storage apparatus is used in its ordinary capacity to store data (MPEP 2106.05(f)(2)).
Limiting each predictor to learn from the training data associated with that predictor does not change this conclusion. The limitation specifies only which stored data each generic predictor uses for learning; it adds no detail about how the learning is technologically accomplished and does not change how any predictor learns or predicts (MPEP 2106.05(f)(1)). The conclusion is the same when the limitation is considered together with the recited determination and accumulation rather than in isolation (MPEP 2106.04(d), subsection III). Routing each observation to the predictor that made the selection decides which data each predictor learns from, but the predictors still learn and predict in the ordinary way (¶[0063]-[0065]). The specification states that this arrangement makes it “possible to cause a predictor used in determination of an option to learn more so that an observation value can be predicted more accurately” (¶[0172]) and to “more effectively cause a predictor that has insufficient prediction accuracy to learn” (¶[0089]). Under Desjardins, that asserted benefit is more accurate prediction for the purpose of making better selections, i.e., an improvement in the mathematical estimation and decision procedure, not in the functioning of a computer or in another technology (MPEP 2106.05(a)). Further, the specification ties the benefit to the third example embodiment, in which the observation used for further learning is obtained by selecting an option “having a large prediction error” (¶[0089]; see also ¶[0069], [0085]). Claim 5, which depends from claim 1 rather than claim 2, recites no prediction error and no such selection rule, and therefore does not reflect the asserted improvement (Intellectual Ventures I v. Symantec, 838 F.3d at 1316). Accordingly, claim 5 does not satisfy Step 2A, Prong 2.
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception. For the same reasons as discussed in the Step 2B analysis of claim 1, the generic apparatus and processor, the acquisition process, the storage apparatus, the accumulation of training data, and the predictors are mere instructions to apply the exception or well-understood, routine, conventional activity. With respect to the narrowed elements, storing records together with an identifier of the predictor used is the well-understood, routine, conventional function of electronic recordkeeping and of storing and retrieving information in memory (Alice, 573 U.S. at 225; Ultramercial, 772 F.3d at 716; Versata, 793 F.3d at 1334; OIP Techs., 788 F.3d at 1363; MPEP 2106.05(d), subsection II). Training each predictor on the data associated with it is the same generic “machine learning using the training data group” that the specification describes without particulars (¶[0063]-[0064]), and the specification describes “learning independently” simply as using training data sets that “differ from each other” (¶[0019]); the limitation remains a mere instruction to apply the exception with generic machine learning (MPEP 2106.05(f)). The ordered combination adds nothing beyond the elements considered individually. Accordingly, claim 5 does not satisfy Step 2B and is rejected under 35 U.S.C. 101.
CLAIM 6
Step 1: Claim 6 depends from claim 1 and recites an information processing apparatus, which falls within the statutory category of a machine. See MPEP 2106.03.
Step 2A, Prong 1: Claim 6 incorporates all the limitations of claim 1 and therefore recites the same abstract idea identified in claim 1. Claim 6 additionally recites that “each of the plurality of predictors learns a linear relation as the relation.” Unlike the generic “learning a relation” of claim 1, this limitation names a specific mathematical relationship (a linear relationship between the relevant information and the observation value), and learning that relation means determining the coefficients of a linear function from the training data (¶[0063]; ¶[0121] (“θt is a coefficient that represents a linear relation learned by machine learning”); ¶[0120] (expression (5) is a mathematical relationship and calculation (MPEP 2106.04(a)(2), subsections I(A) and I(C)). Like the limitation of claim 2 of USPTO Example 47 that recites “a backpropagation algorithm and a gradient descent algorithm,” the limitation sets forth the mathematics by name and therefore recites, rather than merely involves, a mathematical concept. The recitation of “each of the plurality of predictors” refers to the predictors of claim 1, which remain additional elements. No new additional elements are introduced.
Step 2A, Prong 2 and Step 2B: No new additional elements are introduced. Therefore, the analysis from parent claim is maintained.
CLAIM 7
Step 1: Claim 7 recites “An information processing method” comprising acquiring, determining, and accumulating steps, which is a series of acts and therefore falls within the statutory category of a process. See MPEP 2106.03. Accordingly, claim 7 satisfies Step 1 of the eligibility analysis.
Step 2A, Prong 1: Claim 7 recites an abstract idea, namely a mental process. Specifically, claim 7 recites “determining, by the information processing apparatus, an option to be selected from among the plurality of options” and that the predictors are used “in order to determine an option to be selected from among the plurality of options.” For the reasons explained for claim 1, determining which option to select from among candidate options on the basis of information associated with the options and the gains observed for previously selected options is an evaluation and judgment that can practically be performed in the human mind or with pen and paper, and it is a mental process even when performed by a computer. See MPEP 2106.04(a)(2), subsection III; Electric Power Group, 830 F.3d at 1353-54.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application.
For the reasons explained for claim 1 (see USPTO Examples 39 and 47), the limitation “the plurality of predictors independently learning a relation between the relevant information and the observation value with reference to the training data” is not treated as reciting a judicial exception and is evaluated as an additional element. The additional elements are: (a) the “information processing apparatus” that performs each step; (b) “acquiring, by an information processing apparatus, pieces of relevant information which are respectively associated with a plurality of options”; (c) “accumulating, by the information processing apparatus, an observation value of a gain which has been obtained when the option determined had been selected and relevant information of the option in a storage apparatus as training data”; and (d) “any of a plurality of predictors being used,” with “the plurality of predictors independently learning a relation between the relevant information and the observation value with reference to the training data.”
The additional elements, considered individually and as an ordered combination, do not integrate the judicial exception into a practical application. They are the same as the additional elements of claim 1, recited as method steps performed “by an information processing apparatus” rather than by a processor of an apparatus. See MPEP 2106.04(d).
The improvements analysis set forth for claim 1 applies equally to claim 7. The benefit asserted in the specification, a further increase in the accumulation of observation values (¶[0005]-[0006], [0012], [0026]), is an improvement in the abstract selection decision itself, not in the functioning of a computer or in another technology (MPEP 2106.05(a); Trading Technologies, 921 F.3d at 1093-94), and the specification describes no improvement in how the computer operates (contrast Desjardins). In any event, claim 7 does not recite the predictor cascade, prediction errors, thresholds, or dimensionality-based conditions that the specification credits with its demonstrated results (¶[0086]-[0089], [0149]-[0154]), and therefore does not reflect the asserted improvement (Intellectual Ventures I v. Symantec, 838 F.3d at 1316).
Considering each additional element: (a) the information processing apparatus is a generic computer used as a tool (¶[0157]-[0158]; MPEP 2106.05(f)); (b) acquiring the relevant information is mere data gathering and insignificant pre-solution activity (MPEP 2106.05(g); OIP Techs., 788 F.3d at 1363); (c) accumulating the observed gain and relevant information in a storage apparatus as training data is insignificant extra-solution activity in the form of necessary data gathering and storage (MPEP 2106.05(g)); and (d) using any of a plurality of independently learning predictors invokes generic machine learning at a high level of generality as a tool for the determination, without any mechanism for achieving a technological result, which is mere instructions to apply the exception (MPEP 2106.05(f)) and generally links the exception to the field of machine learning (MPEP 2106.05(h); Recentive Analytics, 134 F.4th 1205). As an ordered combination, the elements carry out the abstract decision on a generic computer through conventional data gathering, generic learned prediction, and data storage; they do not improve the computer or any other technology, use a particular machine, transform an article, or otherwise meaningfully limit the exception (MPEP 2106.05(a)-(c), (e)). Accordingly, claim 7 does not satisfy Step 2A, Prong 2.
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception. Elements (a) and (d) are mere instructions to apply the exception (MPEP 2106.05(f)), and that conclusion carries over to Step 2B. Their generic nature is also evidenced by the specification, which describes only ordinary processors and memories (¶[0157]-[0158]) and generic machine learning, “for example” of a linear relation (¶[0063]-[0064]), and which acknowledges that learning a relation from data of previous trials and predicting gains to choose an option was already performed by known contextual bandit algorithms (¶[0003], [0150]). See MPEP 2106.07(a), subsection III(A). Elements (b) and (c), re-evaluated at Step 2B, are well-understood, routine, conventional computer functions of receiving data (buySAFE, 765 F.3d at 1355; OIP Techs., 788 F.3d at 1363), storing and retrieving information in memory (Versata, 793 F.3d at 1334), and electronic recordkeeping (Alice, 573 U.S. at 225; Ultramercial, 772 F.3d at 716). See MPEP 2106.05(d), subsection II; ¶[0018], [0020]. The ordered combination is the same conventional arrangement discussed for claim 1 and provides no inventive concept. Accordingly, claim 7 does not satisfy Step 2B and is rejected under 35 U.S.C. 101.
CLAIM 8
Step 1: Claim 8 recites “A non-transitory storage medium storing a program that is to be executed by a computer.” A non-transitory storage medium falls within the statutory category of an article of manufacture. See MPEP 2106.03. Accordingly, claim 8 satisfies Step 1 of the eligibility analysis.
Step 2A, Prong 1: Claim 8 recites an abstract idea, namely a mental process. Specifically, claim 8 recites “a determination process of determining an option to be selected from among the plurality of options” and specifies that, “in the determination process, an option to be selected from among the plurality of options being determined.” For the reasons explained for claim 1, this determination is an evaluation and judgment that can practically be performed in the human mind or with pen and paper, and it is a mental process even when performed by a computer executing a stored program. See MPEP 2106.04(a)(2), subsection III; Versata, 793 F.3d at 1335.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application.
For the reasons explained for claim 1 (see USPTO Examples 39 and 47), the limitations “with use of any of a plurality of predictors” and “the plurality of predictors independently learning a relation between the relevant information and the observation value with reference to the training data” are evaluated as additional elements rather than as part of the judicial exception. The additional elements are: (a) the “non-transitory storage medium storing a program that is to be executed by a computer,” the program “causing the computer to carry out” the recited processes; (b) “an acquisition process of acquiring pieces of relevant information which are respectively associated with a plurality of options”; (c) “an accumulation process of accumulating, as training data, an observation value of a gain which has been obtained when the option determined in the determination process had been selected and relevant information of the option in a storage apparatus”; and (d) the plurality of independently learning predictors with which the option is determined.
The additional elements, considered individually and as an ordered combination, do not integrate the judicial exception into a practical application. See MPEP 2106.04(d).
The non-transitory storage medium storing a program executed by a computer is a generic computer-readable medium holding instructions to carry out the abstract idea on a generic computer; the specification explains that the program may be stored on “a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like” and executed by a generic processor (¶[0157]-[0160]). Reciting the abstract idea as instructions stored on a medium is no more than mere instructions to apply it on a computer (MPEP 2106.05(f)); media claims that “add nothing of substance to the underlying abstract idea” are ineligible for the same reasons as the corresponding method (Alice, 573 U.S. at 226-27). The remaining additional elements (b) through (d) are the same as those of claim 1, and the analysis of claim 1, including the Desjardins improvements analysis, applies equally: the asserted benefit of greater accumulation of observation values (¶[0005]-[0006], [0012], [0026]) improves the abstract selection decision rather than a computer or other technology (MPEP 2106.05(a)), and claim 8 omits the cascade, prediction-error, threshold, and dimensionality features that the specification credits with its demonstrated results (¶[0086]-[0089], [0149]-[0154]), so it does not reflect the asserted improvement (Intellectual Ventures I v. Symantec, 838 F.3d at 1316). As in claim 1, the acquisition and accumulation processes are insignificant extra-solution data gathering and storage (MPEP 2106.05(g)), and the generically recited predictors are mere instructions to apply the exception that generally link it to machine learning (MPEP 2106.05(f), (h); Recentive Analytics, 134 F.4th 1205). As an ordered combination, the elements carry out the abstract decision on a generic computer through conventional data flow and do not improve any technology. Accordingly, claim 8 does not satisfy Step 2A, Prong 2.
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception. The storage medium, the computer executing the program, and the generically recited predictors are mere instructions to apply the exception (MPEP 2106.05(f)), and that conclusion carries over to Step 2B; their generic nature is also evidenced by the specification's description of ordinary storage media and processors (¶[0157]-[0160]) and of generic machine learning (¶[0003], [0063]-[0064], [0150]). See MPEP 2106.07(a), subsection III(A). Storing a program and data on a medium is the well-understood, routine, conventional function of storing and retrieving information in memory (Versata, 793 F.3d at 1334; OIP Techs., 788 F.3d at 1363), and acquiring the relevant information and accumulating the training data are the well-understood, routine, conventional computer functions of receiving data (buySAFE, 765 F.3d at 1355), storing information in memory (Versata, 793 F.3d at 1334), and electronic recordkeeping (Alice, 573 U.S. at 225; Ultramercial, 772 F.3d at 716). See MPEP 2106.05(d), subsection II; spec ¶[0018], [0020]. The ordered combination is the same conventional arrangement discussed for claim 1 and provides no inventive concept. Accordingly, claim 8 does not satisfy Step 2B and is rejected under 35 U.S.C. 101.
CLAIM 9
Step 1: Claim 9 recites “An information processing system comprising an information processing apparatus recited in claim 1 and a server.” Because it includes all the limitations of claim 1, claim 9 is treated as depending from claim 1. A system comprising an apparatus and a server falls within the statutory category of a machine. See MPEP 2106.03.
Step 2A, Prong 1: Claim 9 incorporates all the limitations of claim 1 and therefore recites the same abstract idea identified in claim 1. The limitations added by claim 9 are performed by “a server”: “an acquisition process of acquiring the relevant information and transmitting the relevant information to the information processing apparatus”; “a selection process of selecting an option which is indicated by information received from the information processing apparatus”; and “an observation process of observing a gain which is obtained by selection in the selection process, and transmitting an observation value which has been observed to the information processing apparatus.” These limitations gather and transmit data, carry out the option that has already been determined, and report the observed result; they are evaluated below as additional elements. The additional elements of claim 9 are those of claim 1 plus the server and its acquisition, selection, and observation processes.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application. For the same reasons as discussed in the Step 2A, Prong 2 analysis of claim 1, the additional elements of claim 1 (the processor, the acquisition and accumulation processes, the storage apparatus, and the plurality of predictors) do not integrate the judicial exception into a practical application.
With respect to the new additional elements, the “server” is a generic computer that communicates with the information processing apparatus over a conventional network such as a LAN, a WAN, a public network, or a mobile data communication network (¶[0029], [0034], [0095]-[0101]); it is used only as a tool (MPEP 2106.05(f)), and distributing the processes between a client apparatus and a server merely confines the abstract idea to a networked computing environment (MPEP 2106.05(h)). The server's “acquisition process” of acquiring and transmitting the relevant information and its “observation process” of observing the gain obtained by the selection and transmitting the observation value are mere data gathering and transmission that supply the inputs to the abstract determination and to the learning (MPEP 2106.05(g)); they are comparable to “[p]resenting offers to potential customers and gathering statistics generated based on the testing about how potential customers responded to the offers,” which was found to be mere data gathering (MPEP 2106.05(g), citing OIP Techs., 788 F.3d at 1363). The “selection process” of selecting the option indicated by the information received from the apparatus merely carries out the result of the abstract determination; in the specification's examples it is performed by transmitting information indicating that the option has been selected to a terminal, or by sending a promotion to the indicated user (¶[0036], [0099]). At this level of generality it is insignificant post-solution activity that does not meaningfully limit how the determination is made (MPEP 2106.05(g)). Unlike the remedial network actions that integrated the exception in claim 3 of USPTO Example 47, the selection process effects no technological change; it acts on whichever option was chosen. As an ordered combination, these elements recite a conventional client-server exchange of data and results that carries out the abstract idea and does not improve the server, the network, or any other technology (MPEP 2106.05(a)). Accordingly, claim 9 does not satisfy Step 2A, Prong 2.
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception. For the same reasons as discussed in the Step 2B analysis of claim 1, the additional elements of claim 1 do not amount to significantly more. The new elements are likewise well-understood, routine, and conventional. A generic server sending and receiving data over a network performs the conventional function of receiving or transmitting data over a network (Intellectual Ventures I v. Symantec, 838 F.3d at 1321; TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016); OIP Techs., 788 F.3d at 1363; buySAFE, 765 F.3d at 1355). As described in the specification, the selection and observation processes consist of transmitting a notice of the selection to a terminal and receiving the result back (¶[0036]-[0037]), or sending a promotion to a user and observing whether the user purchased the product (¶[0099]-[0100]); at the claimed level of generality these are the conventional activities of transmitting and receiving data and of presenting offers and gathering statistics (OIP Techs., 788 F.3d at 1362-63; MPEP 2106.05(d), subsection II). The specification describes the server, its units, the terminals, and the networks only generically (¶[0029], [0034]-[0037], [0095]-[0101]), and the computer hardware generically (¶[0157]-[0159]). See MPEP 2106.07(a), subsection III(A). As an ordered combination, the elements form a conventional client-server arrangement in which a server supplies data, receives a decision, acts on it, and reports the result, and they provide no inventive concept (contrast BASCOM Global Internet Servs. v. AT&T Mobility LLC, 827 F.3d 1341, 1350 (Fed. Cir. 2016), in which an inventive concept was found in a non-conventional and non-generic arrangement of known, conventional pieces). Accordingly, claim 9 does not satisfy Step 2B and is rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 and 5-9 are rejected under 35 USC 103 as being unpatentable over Contextual Bandits with Linear Payoff Functions to Chu et al. (hereinafter Chu) in view of US Pat. Pub. No. 2015/0095271A1 to Ioannidis et al. (hereinafter Ioannidis).
Per claim 1, Chu discloses:
…an acquisition process of acquiring pieces of relevant information which are respectively associated with a plurality of options (Chu: p. 209, Section 3…on every round Chu's learner observes, before choosing, a d-dimensional feature vector for each of the K available actions, which constitutes the acquiring pieces of relevant information respectively associated with a plurality of options limitation under BRI, "On each round t for each action a the learner observes K feature vectors xt,a ∈ Rd, with ∥xt,a∥ ≤ 1, where ∥·∥ denotes the ℓ2-norm").
a determination process of determining an option to be selected from among the plurality of options (Chu: p. 211, Algorithm 3, lines 8 and 14…on each round Chu's SupLinUCB settles on exactly one of the K actions, either the action maximizing the upper confidence bound or an action whose width exceeds the level threshold, which constitutes the determination process limitation under BRI, "Choose at = arg maxa∈Âs (r̂st,a + wst,a)…Choose at ∈ Âs such that wst,at > 2−s").
an accumulation process of accumulating, as training data, an observation value of a gain which has been obtained when the option determined in the determination process had been selected and relevant information of the option… (Chu: p. 209, Section 3…after the chosen action's reward is received, SupLinUCB adds that round to the index set Ψs, and BaseLinUCB forms its regression statistics from the feature vector and the observed reward of every round in that index set, so the reward (gain) observed for the selected action and that action's feature vector are retained as training data, which constitutes the accumulation process limitation under BRI, "After observing the feature vectors the learner then selects an action at and receives reward rt,at"; p. 211, Algorithm 3, line 15…"Update the index sets at all levels: Ψs′t+1 ← Ψs′t ∪ {t}, if s = s′"; p. 210, Algorithm 2, lines 1-2…"At ← Id + Στ∈Ψt x⊤τ,aτ xτ,aτ…bt ← Στ∈Ψt rτ,aτ xτ,aτ").
in the determination process, an option to be selected from among the plurality of options being determined with use of any of a plurality of predictors, and the plurality of predictors independently learning a relation between the relevant information and the observation value with reference to the training data (Chu: p. 210, Section 5…SupLinUCB maintains S = ln T levels, each level s running its own BaseLinUCB linear predictor whose estimate θt is fit only on the rounds in that level's own index set Ψs, and the action is chosen using whichever level's predictor triggers the choice, so the per-level predictors learn the feature-to-reward relation independently of one another from separate training data, which constitutes the plurality of predictors independently learning a relation limitation under BRI, "To handle this problem, we modify the algorithm into BaseLinUCB (Algorithm 2) which assumes statistical independence among the samples, and then use a master algorithm SupLinUCB (Algorithm 3) to ensure the assumption holds"; p. 211, Algorithm 3, lines 1-2 and 6…"S ← ln T…Ψst ← ∅ for all s ∈ [T]…Use BaseLinUCB with Ψst to calculate the width, wst,a, and upper confidence bound, r̂st,a + wst,a, for all a ∈ Âs"; p. 210, Algorithm 2, line 3…"θt ← At−1 bt").
Chu does not expressly disclose, but Ioannidis does teach:
An information processing apparatus, comprising at least one processor, the at least one processor carrying out (Ioannidis: ¶[0047], FIG. 2:220…Ioannidis's learning and selection engine 200 executes its contextual linear bandit algorithm on processor 220 using program instructions held in memory 230, which constitutes the information processing apparatus comprising at least one processor under BRI, "Processor 220 provides computation functions for the learning and selection engine 200"; ¶[0048], FIG. 2:230…"Processor 220 utilizes program memory instructions to execute a method, such as method 400 of FIG. 4, to interpret received requests and data as well as to and to produce arm/selection data for transmission across the network interface 210"):
…in a storage apparatus (Ioannidis: ¶[0045], FIG. 1:130…Ioannidis records the arms played, their contexts, and the corresponding observed rewards in history of events memory 130 for later use by the estimator, which constitutes the accumulating training data in a storage apparatus under BRI, "Also illustrated in FIG. 1 are an arms/actions database 120 and a history of events memory 130 which is a storage area for arm/actions played, corresponding rewards, and contexts").
Chu and Ioannidis are analogous art because they are from the same field of endeavor, specifically contextual multi-armed bandit algorithms that select an action from feature (context) information under a linear reward model and use the observed reward to learn that model. They address the same problem of selecting actions, such as advertisements, so as to maximize cumulative reward.
Before the effective filing date of the claimed invention, it would have been obvious to a PHOSITA to implement Chu's SupLinUCB algorithm on the processor-and-memory learning and selection engine of Ioannidis, executing the algorithm's acquisition, determination, and accumulation steps on Ioannidis's processor 220 and retaining Chu's per-level training data of observed rewards and feature vectors in Ioannidis's history of events memory.
The suggestion/motivation for doing so would have been provided by Chu itself, which teaches that its linear-payoff bandit is directed to practical computerized applications such as online advertisement selection, "An example application for contextual bandits with linear payoffs is the Internet advertisement selection problem [Abe et al., 2003], where advertisement and webpage features are used to construct a linear function to predict the probability of a user clicking on a given advertisement" (Chu: p. 208, Section 1), and that the LinUCB family is readily implemented in software, "LinUCB has a couple of practical advantages over LinRel: first, it is simpler to state and implement; second, it uses ridge regression as its core operation, which is often easier to solve with standard software packages" (Chu: p. 209, Section 2). Ioannidis in turn teaches that a contextual linear bandit for advertisement placement is executed by a processor operating on a stored history of rewards and contexts, "Processor 220 provides computation functions for the learning and selection engine 200" (Ioannidis: ¶[0047]). Furthermore, this is the combination of prior art elements according to known methods to yield predictable results, the rationale of MPEP § 2143(I)(A), because executing a known bandit algorithm on a known processor-and-memory engine yields nothing more than the predictable computerized execution of that algorithm.
Per claim 2, Chu combined with Ioannidis discloses claim 1. Chu further teaches in the determination process, (i) a prediction error is predicted with use of any of the plurality of predictors for each of one or more options included in an option group to be dealt with by the predictor, the prediction error being involved in a prediction value of the gain (Chu: p. 211, Algorithm 3, line 6…at each level s, BaseLinUCB computes for every action in the current candidate set Âs a predicted reward r̂ and a width w that bounds the error of that reward prediction, which constitutes the prediction error involved in a prediction value of the gain under BRI, "Use BaseLinUCB with Ψst to calculate the width, wst,a, and upper confidence bound, r̂st,a + wst,a, for all a ∈ Âs"; p. 211, Section 5.1…"The right-hand side above decomposes the prediction error into a variance term (first) and a bias term (second)");
(ii) in a case where a determination condition pertaining to prediction errors is not satisfied, an option to be selected from among options in an option group dealt with by the predictor is determined (Chu: p. 211, Algorithm 3, lines 7-14…SupLinUCB advances to the next level only when the widths of all candidates at level s are at most 2−s without all being within 1/√T; when that width condition is not met, SupLinUCB instead chooses an action from the current set Âs, either the action maximizing the upper confidence bound (line 8) or an action whose width exceeds 2−s (line 14), which constitutes the determining an option when the determination condition is not satisfied under BRI, "if wst,a ≤ 1/√T for all a ∈ Âs then Choose at = arg maxa∈Âs (r̂st,a + wst,a)…else if wst,a ≤ 2−s for all a ∈ Âs then…else Choose at ∈ Âs such that wst,at > 2−s");
(iii) in a case where the determination condition is satisfied, one or more options are extracted from the option group dealt with by the predictor, and a prediction error of each of the one or more options which have been extracted is predicted, with use of another predictor, as an option group to be dealt with by the another predictor (Chu: p. 211, Algorithm 3, lines 10-12…when the width condition is met at level s, SupLinUCB keeps only the actions whose upper confidence bound is within 21−s of the best, forming the reduced set Âs+1, increments s, and repeats the width calculation for Âs+1 with the next level's BaseLinUCB predictor built on Ψs+1, which constitutes the extracting one or more options and predicting their prediction errors with another predictor under BRI, "Âs+1 ← {a ∈ Âs | r̂st,a + wst,a…≥ maxa′∈Âs (r̂st,a’ + wst,a’) − 21−s}…s ← s + 1"; p. 211, Algorithm 3, lines 5-6 and 17…"repeat Use BaseLinUCB with Ψst to calculate the width…until an action at is found").
Per claim 3, Chu combined with Ioannidis discloses claim 2. Chu further teaches in the determination process, a condition that includes a condition in which each of prediction errors is equal to or lower than a threshold is used as the determination condition (Chu: p. 211, Algorithm 3, line 10…SupLinUCB's level-advancement test requires the width of every remaining action to be at or below the level threshold 2−s, which constitutes the each of prediction errors equal to or lower than a threshold under BRI, " else if wst,a ≤ 2−s for all a ∈ Âs then").
Per claim 5, Chu combined with Ioannidis discloses claim 1. Chu further teaches in the accumulation process, the training data is accumulated…while a piece of training data is associated with a predictor which has been used in determination of an option in the determination process (Chu: p. 211, Algorithm 3, line 15…when an action is chosen at level s because its width exceeds 2−s, round t is added only to the index set Ψs of that same level s while the index sets of all other levels are left unchanged, which constitutes the training data associated with the predictor used in determination of the option under BRI, " Update the index sets at all levels: Ψs′t+1 ← Ψs′t ∪ {t}, if s = s′; Ψs′t, otherwise").
each of the plurality of predictors learns the relation with reference to a piece of training data associated with that predictor from among the training data accumulated… (Chu: p. 210, Algorithm 2, line 0…each level's BaseLinUCB is invoked with that level's own index set Ψst and fits its estimate only on the rounds in that set, which constitutes the each predictor learning with reference to training data associated with that predictor under BRI, "Inputs: α ∈ R+, Ψt ⊆ {1, 2, · · · , t − 1}"; p. 211, Algorithm 3, line 6…"Use BaseLinUCB with Ψst to calculate the width, wst,a, and upper confidence bound, (r̂st,a + wst,a, for all a ∈ Âs").
Per claim 6, Chu combined with Ioannidis discloses claim 1. Chu further teaches each of the plurality of predictors learns a linear relation as the relation (Chu: p. 208, Section 1…each BaseLinUCB predictor estimates a weight vector θt by ridge regression and predicts the reward of an action as the linear function θ⊤t xt,a of its feature vector, which constitutes the linear relation under BRI, "In this contextual bandit setting, the learner competes with the set of all linear predictors on the feature vectors"; p. 210, Algorithm 2, lines 3 and 7…" θt ← At−1 bt … r̂t,a ← θ⊤t xt,a").
Claim 7 is substantially similar in scope and spirit as claim 1. Therefore the rejection of claim 1 is applied accordingly.
Per claim 8, the acquisition, determination, and accumulation processes that the program causes the computer to carry out are substantially similar in scope and spirit as those of claim 1 and are taught by Chu as applied to claim 1 above, with the storage apparatus taught by Ioannidis as applied to claim 1 above.
Chu does not expressly disclose, but Ioannidis does teach:
A non-transitory storage medium storing a program that is to be executed by a computer, the program causing the computer to carry out (Ioannidis: ¶[0048], FIG. 2:230…Ioannidis's memory 230 holds program memory instructions that processor 220 executes to carry out the contextual bandit method, and a physical memory holding executable instructions constitutes the non-transitory storage medium storing a program that is to be executed by a computer under BRI, "Memory 230 can provide the repository for storage of information such as program memory, downloads, uploads, or scratchpad calculations…Processor 220 utilizes program memory instructions to execute a method, such as method 400 of FIG. 4"):
Chu and Ioannidis are analogous art because they are from the same field of endeavor, specifically contextual multi-armed bandit algorithms that select an action from feature (context) information under a linear reward model and use the observed reward to learn that model.
Before the effective filing date of the claimed invention, it would have been obvious to a PHOSITA to store Chu's SupLinUCB algorithm as program instructions in the memory of Ioannidis's learning and selection engine for execution by its processor.
The suggestion/motivation for doing so would have been provided by Chu itself, which teaches that its algorithm is simple to implement in software, "LinUCB has a couple of practical advantages over LinRel: first, it is simpler to state and implement; second, it uses ridge regression as its core operation, which is often easier to solve with standard software packages" (Chu: p. 209, Section 2), and by Ioannidis, which teaches executing a contextual linear bandit from stored program instructions, "Processor 220 utilizes program memory instructions to execute a method, such as method 400 of FIG. 4" (Ioannidis: ¶[0048]). Furthermore, this is the combination of prior art elements according to known methods to yield predictable results, the rationale of MPEP § 2143(I)(A).
Per claim 9, Chu combined with Ioannidis discloses claim 1, which supplies An information processing system comprising an information processing apparatus recited in claim 1…
Chu does not expressly disclose, but Ioannidis does teach:
…and a server, the server carrying out (Ioannidis: ¶[0053], FIG. 3:308…Ioannidis's advertisement placement apparatus 308 is a networked machine that receives user requests over network 306 and works with the learning and selection engine 200', and Ioannidis teaches that these components may be distributed and located in a server, which constitutes the server under BRI, "The search request is communicated to the advertisement placement apparatus 308 having access to the learning and selection engine 200' which is a modified version of learning and selection engine 200 having an additional interface to advertisement database 310"; ¶[0054]…"As is well known, the components of the advertisement placement apparatus can be grouped together as shown or they can be distributed in any manner"; ¶[0062]…"one of skill in the art will recognize that implementation options exist such as distributed functionality of components, consolidation of components, and location in a server as a service to users"):
an acquisition process of acquiring the relevant information and transmitting the relevant information to the information processing apparatus (Ioannidis: ¶[0055]…apparatus 308 receives the user's search request, derives its context information (the search parameters), and passes that context to the learning and selection engine 200', which constitutes the acquiring and transmitting the relevant information under BRI, "The request is processed appropriately with context information, such as the parameters of the search, given to the learning and selection engine 200'");
a selection process of selecting an option which is indicated by information received from the information processing apparatus (Ioannidis: ¶[0060], FIG. 3:310…the arm determined by the engine is played by apparatus 308 selecting the corresponding advertisement from advertisement database 310 and sending it to the user device, which constitutes the selecting an option indicated by information received from the information processing apparatus under BRI, "In the setting of an advertisement placement, as shown in FIG. 3, the arm/action is the selection of an appropriate advertisement from a database of advertisements 310. The selected advertisement is sent to the user device from the advertisement placement apparatus 308 to the user device 302 via the network 306"); and
an observation process of observing a gain which is obtained by selection in the selection process, and transmitting an observation value which has been observed to the information processing apparatus (Ioannidis: ¶[0054], FIG. 3:309…in Ioannidis's FIG. 3 arrangement all traffic between user device 302 and the learning and selection engine 200' passes through network interface 309 of advertisement placement apparatus 308, whose transceiver both sends the placed advertisement and receives the return traffic, so the user's response to the advertisement that apparatus 308 placed reaches the engine through apparatus 308 as the reward; apparatus 308 thereby receives the gain produced by its own selection and passes the observed value on to the engine, which constitutes the observing a gain and transmitting an observation value to the information processing apparatus under BRI, "The search is communicated through the network interface device 304, through network 306, through network interface 309, to advertisement placement apparatus 308"; ¶[0055]…"The selected advertisement is then sent to the user device 302 via the transceiver (receiver/transmitter) of the interface network 309, though the network 306 and network interface device 304"; ¶[0058]…"As part of step 410, the user responds to the advertisements and thus provides rewards back to the learning and selection engine 200' via the network 306"; ¶[0060]…"At step 435, a reward from the user device is received by the learning and selection engine").
The rationale to combine Ioannidis with Chu is the same as the parent claim. As to arranging the server apart from the information processing apparatus, the suggestion is provided by Ioannidis itself, which teaches that the components of its bandit system may be distributed or located in a server and that such arrangements are equivalent, "Although specific architectures are shown for the implementation of a mechanism that performs a contextual multi-armed bandit solution, one of skill in the art will recognize that implementation options exist such as distributed functionality of components, consolidation of components, and location in a server as a service to users" (Ioannidis: ¶[0062]). Furthermore, arranging the advertisement placement server and the learning and selection engine as separate networked apparatuses is the use of a known technique to improve similar devices in the same way, the rationale of MPEP § 2143(I)(C).
Claim 4 is rejected under 35 USC 103 as being unpatentable over Chu in view of Ioannidis, as applied in the rejection of claim 2 above, and further in view of Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits to Li et al. (hereinafter Li).
Per claim 4, Chu combined with Ioannidis discloses claim 2.
Chu combined with Ioannidis does not expressly disclose, but Li does teach:
in the determination process, a condition based on dimensionality of the relevant information is used as the determination condition (Li: p. 5, Algorithm 1, line 8…Li's VCL-SupLinUCB is a modification of Chu's SupLinUCB that keeps the same two width tests at each level, a commit test followed by an advance-and-eliminate test, and Li's commit test compares every width against √(d/T), where d is the dimension of the action feature vectors; because the determination condition of claim 2, as applied above, is formed from those two width tests (Chu: p. 211, Algorithm 3, lines 7 and 10), substituting Li's √(d/T) commit test makes the determination condition depend on the feature dimension d, and Li further sets the number of elimination levels ζ0 from d, which constitutes the condition based on dimensionality of the relevant information under BRI, "if ϖiζ,t ≤ √(d/T) for all i ∈ Nζ,t then"; p. 5, Algorithm 1, lines 10-11…"else if ϖiζ,t ≤ 2−ζ for all i ∈ Nζ,t then Update Nζ+1,t = {i ∈ Nζ,t : x⊤it θ̂ζ,t ≥ maxj∈Nζ,t x⊤jt θ̂ζ,t − 21−ζ}, ζ ← ζ + 1"; p. 5, Algorithm 1, line 1…"Parameters: ζ0 = ⌈log2(√(T/d))⌉"; p. 2, Section 1.1…"There are T ≥ 1 time periods, conveniently denoted as {1, 2, · · · , T}, and a fixed but unknown d-dimensional regression model θ").
Chu, Ioannidis and Li are analogous art because they are from the same field of endeavor, specifically linear contextual multi-armed bandits, and Li expressly builds upon the SupLinUCB algorithm of Chu.
Before the effective filing date of the claimed invention, it would have been obvious to a PHOSITA to replace the fixed 1/√T width test of Chu's SupLinUCB, as implemented on Ioannidis's engine, with Li's dimension-dependent √(d/T) width test and dimension-dependent number of levels, so that the determination condition is based on the dimensionality of the feature information, as claimed.
The suggestion/motivation for doing so would have been provided by Li itself, which presents its algorithm as a direct modification of Chu's SupLinUCB, "The algorithm is a variant of the SupLinUCB algorithm proposed in (Chu et al., 2011), with variable confidence levels at different time periods" (Li: p. 4, Section 2.1), and teaches that the modification tightens the regret guarantee, "Our algorithmic result saves two √log T factors from previous analysis" (Li: p. 1, Abstract). Furthermore, this is the use of a known technique to improve a similar method in the same way, the rationale of MPEP § 2143(I)(C).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN CHEN whose telephone number is (571)272-4143. The examiner can normally be reached M-F 10-7.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN CHEN/Primary Examiner, Art Unit 2125
Prosecution Insights