Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of the Application
The following is a Final Office Action.
In response to Examiner's communication of 1/12/2026, Applicant responded on 4/13/2026. Amended claim 1, 4, 6, 11, 15, 17. Cancelled claims 3, 10.
Claims 1, 2, 4, 7-9, 11, 14-17, and 20 are pending in this application and have been examined.
Response to Amendment
Applicant's amendments to claims 1, 4, 6, 11, 15, 17 are sufficient to overcome the 35 USC 112 rejections set forth in the previous action. The 35 USC 112 rejections are hereby withdrawn.
Applicant's amendments to claims 1, 4, 6, 11, 15, 17 are not sufficient to overcome the 35 USC 101 rejections set forth in the previous action.
Response to Arguments – 35 USC § 101
Applicant’s arguments with respect to the rejections have been fully considered, but they are not persuasive.
Applicant submits, “…These claims define the presently claimed invention as one in which multiple test messages are tied to portions of a web site using cookies. Calculations are performed using an allocated memory space to improve performance, so that a confidence sequence can be continuously updated as responses are detected, and the confidence sequence is displayed as updated. The display can be scrolled for comfortable viewing while the confidence sequence is updated in real time. A user can view the continuously updated display of multiple message treatments and, without waiting for the testing to be completed, select a message to be automatically applied to the web stie for all users by indicating the corresponding test message identifier using the input device… Applicant's amended independent claims do not recite human activities, actions that can be taken by a human, a mental process, or mathematical concepts. The independent claims, as amended, are directed to "automatically publishing the selected test message" to an expanded group of recipients." (Emphasis added.) Further, the user can cut the test short and invoke this automatic publishing at any time that the user appreciates from the live, updated data which messaging and therefore corresponding web site portion performs best. This capability is only made possible by the live, scrollable display and the input device that can indicate the corresponding test message identifier during testing. The best-performing web site portion is automatically published to an expanded group of participants expediently, without waiting for exhaustive data to be produced by extensive…testing. No further action is required by a user of the system. Automatic, early publication of an updated web site is a practical application of the claimed invention. At least these operations require complex electrical signaling. Controlling a display device and responding to input both require complex electrical signaling and circuitry that a human mind cannot replicate. Applicant's claims also recite the use of cookies to cause different test messages to be presented to groups of recipients on the same web site as part of a test. According to the independent claims, the responses to the different messages are automatically compared to a baseline with respect to a response, confidence values are continuously updated as responses are detected, and the confidence values are displayed as updated. The use of cookies to test a web site with different messages while using the display technique discussed above is not an abstract idea, or, in the alternative, is another practical application of any alleged abstract idea. Applicant's claims, as amended, do not recite mental steps….Applicant's claims for the display of continuously updated, live scrollable values in a confidence sequence, along with an identifier that can be selected to automatically publish the selected messaging treatment to a web site, provides a technical solution to a problem that is unique in real-time evaluation of the effects of a messaging treatment. When a different unique test message is provided to different groups of recipients there will be multiple p-values at any given time since these are calculated with respect to a treatment that is a baseline treatment. The confidence sequence is computed for each treatment effect. Confidence bounds are used to estimate a sampling distribution for each treatment effect to restrict and normalize the variance and p-values. The above collection of calculations is computationally intensive and is handled expeditiously enough to provide the real-time display through the use of the allocated storage. See, for example, paragraph [0043] of Applicant's specification. Even so, it can become apparent to the user before the test is complete that a particular message treatment provides the best results, making the rest of the test unnecessary. These claims now recite automatically publishing the selected test message to an expanded group of recipients at any time during the test in response to the selection the corresponding test message identifier using an input device. The claims therefore recite a technical solution to a problem that is unique in its field, making them similar from an eligible subject matter perspective to those in BASCOM Global Internet Services v. AT&T Mobility, 827 F. 3d 1341. Applicant's claimed invention solves the problem of reviewing test results while testing is in progress and still having to wait until the end of the test to act, which would then require the results of large numbers of tests to be compared.…The Office states that, "This this problem does not specifically arise in the realm of computer technology, but rather, this problem existed and was addressed long before the advent of computers. Thus, the claims do not recite a technical improvement to a technical problem or necessarily roots in computing technologies." Office Action, page 6. Applicant respectfully disagrees. The problem is that of reacting to a test of alternative messaging treatments for a web site and expediently publishing a new version of the web site in response to the test. The problem is solved by an automatic publishing capability that can be used at any time during the test. Web sites, messaging treatments for web sites, and web site publishing did not exist "long before the advent of computers." Thus, the claims, as amended, recite a technical solution to a technical problem…Assuming for the sake of argument that the present claims recite an abstract idea, the claims integrate the abstract idea into a practical application. Applicant's amended independent claims include displaying the confidence sequence for each of multiple test messages in visual alignment on a display device while the confidence sequence is being updated. The display device is configured to be scrollable while simultaneously updating the values in the confidence sequence. A user can view each record in conjunction with a test message identifier, can continuously review current confidence values as well as additional information, even while continuously scrolling through large numbers of test results that are displayed as part of the sequence. The user can also select a test messaging treatment that corresponds to the test message identifier, which in turn corresponds to a portion of a web site, and automatically publish the web site with the selected messaging treatment at any time, including prior to completion of the test…” The Examiner respectfully disagrees.
While Applicant’s amendments further prosecution, unlike BASCOM, the claims and the argued elements, are directed to, …evaluation of the effects of a messaging treatment. When a different unique test message is provided to different groups of recipients there will be multiple p-values at any given time since these are calculated with respect to a treatment that is a baseline treatment. The confidence sequence is computed for each treatment effect. Confidence bounds are used to estimate a sampling distribution for each treatment effect to restrict and normalize the variance and p-values…, which is a problem reciting mental process (i.e. humans mentally testing marketing messages to other humans using statistics testing and publishing the most effective marketing message from statistical testing while reviewing statistical testing results on paper), organizing human activities (i.e. humans mentally testing marketing messages to other humans using statistics testing and publishing the most effective marketing message from statistical testing while reviewing statistical testing results on paper), mathematical concepts (i.e. humans mentally testing marketing messages to other humans using statistics testing and publishing the most effective marketing message from statistical testing while reviewing statistical testing results on paper), as established in Step 2A Prong 1. This problem does not specifically arise in the realm of computer technology, but rather, this problem existed and was addressed long before the advent of computers. Thus, the claims do not recite a technical improvement to a technical problem. Additionally, pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components, i.e. computer, display configured to scroll . Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer and graphical user interface, performing extra solution activities. Therefore, as a whole, the additional elements do not integrate the abstract ideas into a practical application in Step 2A Prong 2 (apply it and general link) or amount to significantly more in Step 2B (apply it and WURC).
Even novel and newly discovered judicial exceptions are still exceptions, despite their novelty. July 2015 Update, p. 3; see SAP America Inc. v. Investpic, LLC, No. 2017-2081, slip op. at 2 (Fed Cir. May 15, 2018).
Simply reciting specific limitations that narrow the abstract idea does not make an abstract idea non-abstract. 79 Fed. Reg. 74631; buySAFE Inc. v. Google, Inc., 765 F.3d 1350, 1355 (2014); see SAP America at p. 12. As discussed in SAP America, no matter how much of an advance the claims recite, when “the advance lies entirely in the realm of abstract ideas, with no plausibly alleged innovation in the non-abstract application realm,” “[a]n advance of that nature is ineligible for patenting.” Id. at p. 3.
Claims can recite a mental process even if they are claimed as being performed on a computer. The Supreme Court recognized this in Benson, determining that a mathematical algorithm for converting binary coded decimal to pure binary within a computer’s shift register was an abstract idea. The Court concluded that the algorithm could be performed purely mentally even though the claimed procedures “can be carried out in existing computers long in use, no new machinery being necessary.” 409 U.S at 67, 175 USPQ at 675. See also Mortgage Grader, 811 F.3d at 1324, 117 USPQ2d at 1699 (concluding that concept of “anonymous loan shopping” recited in a computer system claim is an abstract idea because it could be “performed by humans without a computer”).
Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015).
TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea.
Performing a mental process in a computer environment. An example of a case identifying a mental process performed in a computer environment as an abstract idea is Symantec Corp., 838 F.3d at 1316-18, 120 USPQ2d at 1360. In this case, the Federal Circuit relied upon the specification when explaining that the claimed electronic post office, which recited limitations describing how the system would receive, screen and distribute email on a computer network, was analogous to how a person decides whether to read or dispose of a particular piece of mail and that “with the exception of generic computer-implemented steps, there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper”. 838 F.3d at 1318, 120 USPQ2d at 1360. Another example is FairWarning IP, LLC v. Iatric Sys., Inc., 839 F.3d 1089, 120 USPQ2d 1293 (Fed. Cir. 2016). The patentee in FairWarning claimed a system and method of detecting fraud and/or misuse in a computer environment, in which information regarding accesses of a patient’s personal health information was analyzed according to one of several rules (i.e., related to accesses in excess of a specific volume, accesses during a pre-determined time interval, or accesses by a specific user) to determine if the activity indicates improper access. 839 F.3d. at 1092, 120 USPQ2d at 1294. The court determined that these claims were directed to a mental process of detecting misuse, and that the claimed rules here were “the same questions (though perhaps phrased with different words) that humans in analogous situations detecting fraud have asked for decades, if not centuries.” 839 F.3d. at 1094-95, 120 USPQ2d at 1296.
[I]t is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology. For example, in Trading Technologies Int’l v. IBG, 921 F.3d 1084, 1093-94, 2019 USPQ2d 138290 (Fed. Cir. 2019), the court determined that the claimed user interface simply provided a trader with more information to facilitate market trades, which improved the business process of market trading but did not improve computers or technology.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 2, 4, 7-9, 11, 14-17, and 20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 1 recite, “A method comprising:
determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages;
publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment,
testing, over time, using a response module and the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages;
deriving, iteratively over time, using a difference module and the allocated storage, a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric corresponding to the baseline message treatment;
estimating, iteratively over time, using a variance module, a variance of an average of the metric for the plurality of test messages;
calculating a current confidence value using a confidence module, iteratively over time to produce a confidence sequence in the allocated storage, the current confidence value based on the variance and an error-corrected p-value normalized within confidence bounds, wherein the current confidence value corresponds to a current difference value for the comparative difference;
dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …;
…, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and
automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the …..”
Claim 8 recite, “… to perform operations of
determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, and dynamically displaying a confidence sequence:
a … configured to publish the varied portions of the … for a specified period,
a response module configured to test, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages;
a … configured to derive, iteratively over time, using the allocated storage, a comparative lift for an assayed value of the metric for the message response relative a baseline value of the metric corresponding to the baseline message treatment;
a … configured to estimate, iteratively over time, a variance of an average of the metric for the plurality of test messages;
a … configured to calculate, iteratively over time to produce a confidence sequence in the allocated storage, a current confidence value based on the variance and an error-corrected p-value, the current confidence value to produce the confidence sequence; and
an … to, while updating over time, dynamically display the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …, and …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time;
wherein the … automatically publishes, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the….”
Claim 15 recite, “… to perform operations comprising:
determining portions of a … to provide a plurality of test messages;
customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages;
publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …;
testing, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages;
a step for producing, iteratively over time, a current confidence value to produce a confidence sequence in allocated storage, the current confidence value corresponding to a current difference value for a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric; and
dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …;
…, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and
automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the….“
Analyzing under Step 2A, Prong 1:
The limitations regarding, … determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages; publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, testing, over time, using a response module and the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; deriving, iteratively over time, using a difference module and the allocated storage, a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric corresponding to the baseline message treatment; estimating, iteratively over time, using a variance module, a variance of an average of the metric for the plurality of test messages; calculating a current confidence value using a confidence module, iteratively over time to produce a confidence sequence in the allocated storage, the current confidence value based on the variance and an error-corrected p-value normalized within confidence bounds, wherein the current confidence value corresponds to a current difference value for the comparative difference; dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …; …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the … determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, and dynamically displaying a confidence sequence: a … configured to publish the varied portions of the … for a specified period, a response module configured to test, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; a … configured to derive, iteratively over time, using the allocated storage, a comparative lift for an assayed value of the metric for the message response relative a baseline value of the metric corresponding to the baseline message treatment; a … configured to estimate, iteratively over time, a variance of an average of the metric for the plurality of test messages; a … configured to calculate, iteratively over time to produce a confidence sequence in the allocated storage, a current confidence value based on the variance and an error-corrected p-value, the current confidence value to produce the confidence sequence; and an … to, while updating over time, dynamically display the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …, and …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; wherein the … automatically publishes, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the… determining portions of a … to provide a plurality of test messages; customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages; publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …; testing, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; a step for producing, iteratively over time, a current confidence value to produce a confidence sequence in allocated storage, the current confidence value corresponding to a current difference value for a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric; and dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …; …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the…, under the broadest reasonable interpretation, can include a human using their mind and using pen and paper to perform the identified limitations above; therefore, the claims recite a mental process.
Further, …determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages; publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, testing, over time, using a response module and the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; deriving, iteratively over time, using a difference module and the allocated storage, a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric corresponding to the baseline message treatment; estimating, iteratively over time, using a variance module, a variance of an average of the metric for the plurality of test messages; calculating a current confidence value using a confidence module, iteratively over time to produce a confidence sequence in the allocated storage, the current confidence value based on the variance and an error-corrected p-value normalized within confidence bounds, wherein the current confidence value corresponds to a current difference value for the comparative difference; dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …; …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the … determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, and dynamically displaying a confidence sequence: a … configured to publish the varied portions of the … for a specified period, a response module configured to test, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; a … configured to derive, iteratively over time, using the allocated storage, a comparative lift for an assayed value of the metric for the message response relative a baseline value of the metric corresponding to the baseline message treatment; a … configured to estimate, iteratively over time, a variance of an average of the metric for the plurality of test messages; a … configured to calculate, iteratively over time to produce a confidence sequence in the allocated storage, a current confidence value based on the variance and an error-corrected p-value, the current confidence value to produce the confidence sequence; and an … to, while updating over time, dynamically display the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …, and …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; wherein the … automatically publishes, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the… determining portions of a … to provide a plurality of test messages; customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages; publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …; testing, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; a step for producing, iteratively over time, a current confidence value to produce a confidence sequence in allocated storage, the current confidence value corresponding to a current difference value for a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric; and dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …; …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the…, under the broadest reasonable interpretation, are humans mentally testing marketing messages to other humans using statistics testing and publishing the most effective marketing message from statistical testing while reviewing statistical testing results on paper, therefore it is, commercial interactions and managing interactions between people. Thus, the claims recite certain methods of organizing human activity.
Additionally, … determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages; publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, testing, over time, using a response module and the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; deriving, iteratively over time, using a difference module and the allocated storage, a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric corresponding to the baseline message treatment; estimating, iteratively over time, using a variance module, a variance of an average of the metric for the plurality of test messages; calculating a current confidence value using a confidence module, iteratively over time to produce a confidence sequence in the allocated storage, the current confidence value based on the variance and an error-corrected p-value normalized within confidence bounds, wherein the current confidence value corresponds to a current difference value for the comparative difference; dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …; …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the … determining portions of a … to provide a plurality of test messages, customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages, wherein each of the varied portions of the … corresponds to a …, allocating storage for computations with respect to a baseline message treatment, and dynamically displaying a confidence sequence: a … configured to publish the varied portions of the … for a specified period, a response module configured to test, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; a … configured to derive, iteratively over time, using the allocated storage, a comparative lift for an assayed value of the metric for the message response relative a baseline value of the metric corresponding to the baseline message treatment; a … configured to estimate, iteratively over time, a variance of an average of the metric for the plurality of test messages; a … configured to calculate, iteratively over time to produce a confidence sequence in the allocated storage, a current confidence value based on the variance and an error-corrected p-value, the current confidence value to produce the confidence sequence; and an … to, while updating over time, dynamically display the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …, and …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; wherein the … automatically publishes, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the… determining portions of a … to provide a plurality of test messages; customizing the portions of the … to create the plurality of test messages by producing varied portions of the … corresponding to each of the test messages; publishing the varied portions of the … for a specified period, wherein each of the varied portions of the … corresponds to a …; testing, over time, using the …, a metric corresponding to a message response from an independent group of recipients for each of the test messages; a step for producing, iteratively over time, a current confidence value to produce a confidence sequence in allocated storage, the current confidence value corresponding to a current difference value for a comparative difference between an assayed value of the metric for the message response and a baseline value of the metric; and dynamically displaying, while updating over time and using an interface module, the confidence sequence from the allocated storage including the current confidence value and a corresponding test message identifier for each of the plurality of test messages in visual alignment on a …; …, the display of the confidence sequence while simultaneously updating the display of the confidence sequence over time; and automatically publishing, in response to the …, at any time up to conclusion of the testing, a final version of the … including a portion from among the varied portions, the portion corresponding to a selected test message from among the plurality of test messages displayed while …, the selected test message identified by the corresponding test message identifier and indicated using the…, are mathematical concepts.
Accordingly, the claims recite a mental process, certain methods of organizing human activity, mathematical concepts, under the first prong of Step 2A.
Analyzing under Step 2A, Prong 2:
This judicial exception is not integrated into a practical application under the second prong of Step 2A.
In particular, the claims recite the additional elements beyond the recited abstract idea identified under Step 2A, Prong 1, such as:
Claim 1, 8, 15: A system comprising: a memory component; a processing device coupled to the memory component, response module, difference module, variance module, confidence module, A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device, interface module, messaging server, display device, display device is configured to be scrollable, web site, cookie. Server, scrolling, in response to an input device, automatically publishing, in response to the input device, at any time up to conclusion of the testing, a final version of the web site, scroll, in response to an input device, server automatically publishes, in response to the input device… a final version of the web site, scrolling, scrolling, in response to an input device, automatically publishing, in response to the input device
, and pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components.
Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer, website design, tracking humans with cookies and scrollable graphical user interface.
Additionally, with respect to, “transmitting…”, “…publishing…”, “testing…”, “…storage for computations…”, “displaying… scrolling”, these elements do not add a meaningful limitations to integrate the abstract idea into a practical application because they are extra-solution activity, pre and post solution activity - i.e. data gathering – “transmitting…”, “…publishing…”, “testing…”, “…storage for computations…”, data output – “…publishing…”, “displaying…scrolling…”
Analyzing under Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under Step 2B.
As noted above, the aforementioned additional elements beyond the recited abstract idea are not sufficient to amount to significantly more than the recited abstract idea because, as an order combination, the additional elements are no more than mere instructions to implement the idea using generic computer components (i.e. apply it).
Additionally, as an order combination, the additional elements append the recited abstract idea to well-understood, routine, and conventional activities in the field as individually evinced by the applicant’s own disclosure, as required by the Berkheimer Memo, in at least:
[0021] The use of corrected p-values normalized to be within confidence bounds provides a process that produces statistically valid, current confidence values irrespective of the number of messaging treatments being simultaneously tested. The confidence values are updated over time, and current values can be observed at any time during an experiment. The differences between treatments and the confidence values can be displayed in alignment with each other and scrolled as required by the number of messaging treatments being tested, while the values displayed are iteratively updated over time.
[0023] In some examples, the analytics application can be configured to display valid current confidence values and current difference values along with test message identifiers in visual alignment on a display device. The display device can be configured by the analytics application to be scrollable while being iteratively updated over time with the sequence of values.
[0031] Still referring to FIG. 1, in this example, the analytics application 102 includes the difference module 111 for determining current differences for messaging treatments relative to a baseline, the response module 112 for assaying current response metric values, the variance module 114 for determining current variances, the confidence module 120 for calculating confidence values, and the stored p-values 122 being used to derive the confidence values. Analytics application 102 also includes an interface module 130. In some embodiments, the analytics application 102 uses input from input device 140 to configure information displayed on presentation device 108, or to configure, start, or stop a test. The analytics application 102 can render a dynamic display 136 to be output to presentation device 108. For example, the dynamic display may include a window with current confidence values, difference values, and test message identifiers configured to be in visual alignment on presentation device 108, so that data on many messaging treatments can be selectively observed by scrolling through messaging treatments using input device 140, while the data is being iteratively updated over time.
[0036] At block 212 the computing device displays, dynamically while updating over time, at least the current confidence value and the current difference value. For example, a dynamic display 136 can be output through the interface module 130 to the presentation device 108. In some examples, the displayed output at block 212 of process 200 is scrollable and can be updated while scrolling for optimized viewing and review of the valid confidence sequences for each treatment. These confidence values can be observed at any time during the experiment; the test does not need to be completed for valid confidence values to be displayed. A screenshot of an example display will be discussed below with respect to FIG. 6.
[0051] In this example, at least the current confidence value, and the current difference value are displayed, and values are updated so that confidence values are sequentially displayed until the end of the experiment. In some examples, a display window or GUI can be scrolled to view messaging treatments and the values can continue to be updated. In some examples, the analytics application can be configured to display valid current confidence values and difference values along with test message identifiers in visual alignment on a display device such as display device such as presentation device 108. The display device can be configured by the analytics application to be scrollable while being iteratively updated over time with the sequence of values.
[0054] Staying with FIG. 9, area906 lists the pages, rows, and messages displayed for the experiment. In this particular example, there is one page of messaging treatments totaling80 rows. Thus, 80 messages are being tested, including the baseline message. Currently, statistics for messaging treatments one through five are being displayed. Scroll bar908 provides the capability to selectively scroll through results of the test in process while statistics are being updated over time. The analytics application can display these confidence values and other values sequentially over time along with the current difference, or "lift," updating all of these values while maintaining the accuracy of the values and providing a visual display that can be scrolled to provide for examination of the values for all treatments as they are updated while the experiment proceeds.
[0055] FIG. 10 is a diagram of an example of a computing system that can provide anytime-valid confidence sequences when testing multiple messaging treatments according to certain embodiments. System1000 includes a processing device1002 communicatively coupled to one or more memory devices. The processing device1002 executes computer-executable program code stored in the memory component1004. Examples of the processing device1002 include a microprocessor, an application-specific integrated circuit ("ASIC"), a field- programmable gate array ("FPGA"), or any other suitable processing device. The processing device1002 can include any number of processing devices, including a single processing device. The memory component1004 includes any suitable non-transitory computer-readable medium for storing data, program code instructions, or both. A computer-readable medium can include any electronic, optical, magnetic, or other storage device capable of providing a processor with computer-readable, executable instructions or other program code. The memory component can include multiple memory devices to provide a computer-readable medium. Non-limiting examples of a computer-readable medium include a magnetic disk, a memory chip, a ROM, a RAM, an ASIC, optical storage, magnetic tape or other magnetic storage, or any other medium from which a processing device can read instructions. The instructions may include processor- specific instructions generated by a compiler or an interpreter from code written in any suitable computer-programming language, including, for example, C, C++, C#, Visual Basic, Java, Python, Perl, and JavaScript.
[0058] Staying with FIG. 10, in some embodiments, the computing system1000 also includes the presentation device1015. A presentation device1015 can include any device or group of devices suitable for providing visual, auditory, or other suitable sensory output. In examples, presentation device1015 provides the dynamic display of anytime-valid confidence sequences for multiple treatments. Non-limiting examples of the presentation device1015 include a touchscreen, a monitor, a separate mobile computing device, etc. In some aspects, the presentation device1015 can include a remote client-computing device that communicates with the computing system1000 using one or more data networks. System1000 may be implemented as a unitary computing device, for example, a notebook or mobile computer. Alternatively, as an example, the various devices included in system1000 may be distributed and interconnected by interfaces or a network with a central or main computing device including one or more processors.
[0059] Numerous specific details are set forth herein to provide a thorough understanding of the claimed subject matter. However, those skilled in the art will understand that the claimed subject matter may be practiced without these specific details. In other instances, methods, apparatuses, or systems that would be known by one of ordinary skill have not been described in detail so as not to obscure claimed subject matter.
[0060] Unless specifically stated otherwise, it is appreciated that throughout this specification discussions utilizing terms such as "generating,""assaying,""processing,""computing,""determining," and "identifying" or the like refer to actions or processes of a computing device, such as one or more computers or a similar electronic computing device or devices that manipulate or transform data represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of the computing platform.
[0061] The system or systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device can include any suitable arrangement of components that provide a result conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems accessing stored software that programs or configures the computing system from a general-purpose computing apparatus to a specialized computing apparatus implementing one or more implementations of the present subject matter. Any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein in software to be used in programming or configuring a computing device
[0064] Where devices, systems, components or modules are described as being configured to perform certain operations or functions, such configuration can be accomplished, for example, by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation such as by executing computer instructions or code, or processors or cores programmed to execute code or instructions stored on a non-transitory memory medium, or any combination thereof. Processes can communicate using a variety of techniques including but not limited to conventional techniques for inter-process communications, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting.
[0065] While the present subject matter has been described in detail with respect to specific embodiments thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, it should be understood that the present disclosure has been presented for purposes of example rather than limitation and does not preclude inclusion of such modifications, variations, and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art.
Furthermore, as an ordered combination, these elements amount to generic computer components receiving or transmitting data over a network, performing repetitive calculations, electronic record keeping, and storing and retrieving information in memory, which, as held by the courts, are well-understood, routine, and conventional. See MPEP 2106.05(d).
Moreover, the remaining elements of dependent claims do not transform the recited abstract idea into a patent eligible invention because these remaining elements merely recite further abstract limitations that provide nothing more than simply a narrowing of the abstract idea recited in the independent claims.
Looking at these limitations as an ordered combination adds nothing additional that is sufficient to amount to significantly more than the recited abstract idea because they simply provide instructions to use a generic arrangement of generic computer components to “apply” the recited abstract idea, perform insignificant extra-solution activity, and generally link the abstract idea to a technical environment. Thus, the elements of the claims, considered both individually and as an ordered combination, are not sufficient to ensure that the claim as a whole amounts to significantly more than the abstract idea itself. Since there are no limitations in these claims that transform the exception into a patent eligible application such that these claims amount to significantly more than the exception itself, claims 1, 2, 4, 7-9, 11, 14-17, and 20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PO HAN MAX LEE whose telephone number is (571)272-3821. The examiner can normally be reached on Mon-Thurs 8:00 am - 7:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on (571) 272-6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PO HAN LEE/Primary Examiner, Art Unit 3623
Prosecution Insights