DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed on August 29, 2025 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Acknowledgements
This Office Action is in response to Applicant’s responses filed on August 29, 2025 (“August 2025 Response”) which contained, inter alia, claim amendments (“August 2025 Claim Amendments”), and “REMARKS” (“August 2025 Remarks”).
Claims 1-12 and 26 are currently pending and have been examined.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-12 and 26 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
In particular, Claim 1 recites:
wherein the test deployment and data collection interface is configured to select a subset of the plurality of tests for simultaneous execution and ensure that the subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests in a management phase,
However, the original disclosure is silent with regards to “select a subset of the plurality of tests for simultaneous execution and ensure that the subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests.” Although, the specification discusses “orthogonal” in paragraph 27, the specification does not lexicographically define the term “orthogonal.” Furthermore, although the specification discusses “the user interface 1000 depicts two orthogonal tests (depicted as ‘color change’ and ‘ad on homepage’) performed concurrently on different portions of an application or website, with separate sales lift depicted. Such analysis can be achieved via a regression analysis of the interaction data, with confidence intervals being calculated in a variety of ways (as described briefly above), and possibly calculated differently for digital and in-person interactions” in paragraph 52, there is no discussion of selecting a subset of tests from a plurality, nor is there support for simultaneous execution of tests which are determined by the test deployment and data collection interface to be orthogonal previous to execution. Claims 2-12 and 26 recite the same language as Claim 1; therefore, they are rejected for the same reasons given above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6-9, and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Glover et al. (US 2017/0017989 A1)(“Glover”) in view of Apple et al. (US 2007/0038516 Al)(“Apple”) and further in view of Brooks et al. (US 2010/0174671 A1)(“Brooks”).
As to Claim 1, Glover discloses a multivariate testing platform comprising:
a computing device (“a general-purpose computers” [0054]) executing instructions stored in a memory (“a computer readable medium” [0054]), the computing device being caused to:
define a test definition environment (“plurality of test promotions may be designed and tested on a plurality of groups of consumers” [0057], environment that includes web, point-of-sale, Fig.5, and mobile devices, [0117], [0119]) through which sales of products or services occur (“industry that offers goods or services to a buyer” [0010]) including a test deployment and data collection interface (“user interface,” [0130], see also [0063], [0071], [0081], [0128]) configured to:
manage test deployments by the test definition online environment to a controlled subset of users (“subpopulation,” [0059]) across digital and non-digital content delivery channels (“channel (e.g., email versus paper coupon versus notification in loyalty account)” [0116], “Although grocery stores and other brick-and-mortar businesses are discussed in various examples herein, it is expressly contemplated that embodiments of the invention apply also to online shopping and online advertising/promotion and online members/customers.” [0083])([0063], [0066], see also Fig.7), the content delivery channels including all of:
a digital, web-based content delivery channel for distributing the test deployment online (“Generally speaking, test promotions (or generalized public promotions) are presented to consumers as offers. These offers are often online offers and may take the form of, for example, digital coupons or click-through advertisements to be presented on a webpage, mobile app, text/display ads, digital coupons, digital billboard, print-at home coupon, digital wallet, etc.” [0120]);
a digital, mobile app-based content delivery channel for distributing the test on a mobile application (“Generally speaking, test promotions (or generalized public promotions) are presented to consumers as offers. These offers are often online offers and may take the form of, for example, digital coupons or click-through advertisements to be presented on a webpage, mobile app, text/display ads, digital coupons, digital billboard, print-at home coupon, digital wallet, etc.” [0120]); and
a non-digital, point of sale content delivery channel for distributing the test at a physical retail location (“test promotions may be mailed to the individuals” “Redemption may take place using, for example, a printed coupon (which may be mailed or may be printed from an electronic version of the coupon) at the point of sale terminal,” [0114], “Module 812 may represent, for example, the point of sale terminal in a store, a shopping basket on an online shopping website, an app on a smart phone, a webpage displayed on a computer, a social media news feed, etc. where user responses can be received.” [0117], see also Fig.5 which shows the different channels include web, point-of-sale content delivery channel); and
receive test data from an analytics server (analytics engine 132) regarding user interaction regarding the test deployments (“Analysis engine 132 represents a software engine for analyzing the consumer responses to the test promotions.” [0097], [0107], see Fig.2A wherein engine 132 sends data to 160 and 130),
wherein the test deployment and data collection interface is configured to deploy a plurality of tests simultaneously across the digital, web-based content delivery channel, the digital, mobile app-based content delivery channel, and the non-digital, point of sale content delivery channel (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “Generally speaking, each individual test promotion may be designed to test one or more test promotion variables. These test promotions variables may relate to, for example, the size, shape, color, manner of display, manner of discount, manner of publicizing, manner of dissemination pertaining to the goods/services being promoted” [0061]),
wherein the plurality of tests are defined within the test definition retail environment as test proposals having a defined set of characteristics including a test type (“manner of discount” [0061]), a definition of a user population (“subpopulation,” [0059]), a duration (“date,” [0076]), a proposed change (“change,” [0092]), and a selection of one or more channels for test deployment from among the plurality of digital and non-digital purchase channels (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “Generally speaking, each individual test promotion may be designed to test one or more test promotion variables. These test promotions variables may relate to, for example, the size, shape, color, manner of display, manner of discount, manner of publicizing, manner of dissemination pertaining to the goods/services being promoted” [0061]),
wherein the test deployment and data collection interface is configured to select a subset of the plurality of tests for simultaneous execution (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “…promotion testing may be iterated over and over with different subpopulations (segmented using the same or different segmenting criteria) and different test promotions (devised using the same or different combinations of test promotion variables) in order to validate one or more the test promotion response analysis result(s) prior to the formation of the generalized public promotion” [0066], “test promotions may be executed iteratively and/or in a continual fashion on different purposefully segmented subpopulations using different combinations of test promotion variables to continue to obtain insights into consumer actual revealed preferences, even as those preferences change over time.” [0071], “In one or more embodiments, once a set of concepts is developed, the technology advantageously a) will constrain offers to only test ‘viable promotions’, i.e., those that don't violate local laws, conflict with branding guidelines, lead to unprofitable concepts that wouldn't be practically relevant, can be executed on a retailers' system, etc., and/or b) link to the design of experiments for micro-testing to determine which combinations of variables to test at any given time” [0080], “In an embodiment, it is envisioned that dozens, hundreds, or even thousands of these test promotions may be administered concurrently or staggered in time to the dozens, hundreds or thousands of segmented subpopulations. Further, the large number of test promotions executed (or iteratively executed) improves the statistical validity of the correlations ascertained by analysis engine” [0101]) in a management phase (“promotional idea module,” [0079]),
wherein the plurality of tests includes a first test having a first set of characteristics and a second test having a second set of characteristics (“One or more of the test promotion variables may vary from test promotion to test promotion. In the example of FIG. 2A, test promotion 102a involves test variable X1 (representing a given value or attribute for test variable X) while test promotion 102b involves test variable X2 (representing a different value or attribute for test variable X). A test promotion may vary, relative to another test promotion, one test promotion variable (as can be seen in the comparison between test promotions 102a and 102b) or many of the test promotion variables (as can be seen in the comparison between test promotions 102a and 102d).” [0087]),
wherein the first set of characteristics and second set of characteristics are determined to be orthogonal to one another such that the first test and second test are simultaneously executed (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “…promotion testing may be iterated over and over with different subpopulations (segmented using the same or different segmenting criteria) and different test promotions (devised using the same or different combinations of test promotion variables) in order to validate one or more the test promotion response analysis result(s) prior to the formation of the generalized public promotion” [0066], “test promotions may be executed iteratively and/or in a continual fashion on different purposefully segmented subpopulations using different combinations of test promotion variables to continue to obtain insights into consumer actual revealed preferences, even as those preferences change over time.” [0071], “In one or more embodiments, once a set of concepts is developed, the technology advantageously a) will constrain offers to only test ‘viable promotions’, i.e., those that don't violate local laws, conflict with branding guidelines, lead to unprofitable concepts that wouldn't be practically relevant, can be executed on a retailers' system, etc., and/or b) link to the design of experiments for micro-testing to determine which combinations of variables to test at any given time” [0080], “In an embodiment, it is envisioned that dozens, hundreds, or even thousands of these test promotions may be administered concurrently or staggered in time to the dozens, hundreds or thousands of segmented subpopulations. Further, the large number of test promotions executed (or iteratively executed) improves the statistical validity of the correlations ascertained by analysis engine” [0101]) by the test deployment and data collection interface and advance to an execution phase (“the test promotions may be generated using automated test promotion generation software 110, which varies for example the test promotion variables and/or the values of the test promotion variables and/or the number of the test promotion variables to come up with different test promotions.” [0088]).
Glover does not directly disclose:
each user having an assigned global identifier usable across multiple digital devices;
receive test data via a plurality of digital and non-digital purchase channels, wherein the test data is linked to the assigned global identifiers of the subset of users; and
to ensure that the subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests.
Apple teaches
each user having an assigned global identifier usable across multiple digital devices (“Further, by utilizing an Anonymous User ID Code that excludes any personally identifiable information, the present systems and methods can enable an Advertiser to track if an Advertisement delivered to any Media Device 0110 results in a sale in a Retail Channel without utilizing personally identifiable information about an user.” [0719]);
receive test (“The Independent Variable can be any feature whose effect on User Response an Advertiser wants to test.” [0932]) data (“An Advertiser can apply Method 6400 to measure the effect of the Independent Variable on the User Response through a variety of methods. For example, an Advertiser may transmit Advertisement A to Sample Group A and Advertisement B to Sample Group B, where Advertisement A and Advertisement B were equal in every respect, Sample Group A and Sample Group B were statistically similar, and Advertisement A had a first value of the Independent Variable and Advertisement B had a second value of the Independent Variable.” [0943]) via a plurality of digital and non-digital purchase channels (“Retail Channel is any type of channel through which an user can purchase an Product, including, but not limited to: a physical retailer…an online retailer” [0236], “to determine if the difference in ROI due to the effect of the Independent Variable on User Response is statistically significant.” [0942]), wherein the test data is linked to the assigned global identifiers of the subset of users (“Further, by utilizing an Anonymous User ID Code that excludes any personally identifiable information, the present systems and methods can enable an Advertiser to track if an Advertisement delivered to any Media Device 0110 results in a sale in a Retail Channel without utilizing personally identifiable information about an user.” [0719], “By importing data on the retail channel in which a customer bought a product, the report can enable an Advertiser/Manufacturer to measure the ROI of an Advertisement viewed through any Media Device 0110, regardless of the retail channel.” [1150], “Examples of Media Devices 0110 can include…a Print publication 0312, Product Package 0314” [0264]),
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Glover by the features of Apple and in particular to include in Glover, the features of: each user having an assigned global identifier usable across multiple digital devices; and receive test data via a plurality of digital and non-digital product purchase channels, wherein the test data is linked to the assigned global identifiers of the subset of users, as taught by Apple.
A person having ordinary skill in the art would have been motivated to combine these features because it would help to optimize the promotions thereby helping to increase sales and associated profits of the products or services (Glover, [0009]-[0010]).
Brooks teaches ensuring that a subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests (“The methodology disclosed in this application also allows the ability to simultaneously test multiple independent variables during the same time-slot samples, providing that the content associated with the different independent variables is unrelated. This is because experimental content for one independent variable can be control content for another independent variable.” [0434], “The algorithm of FIG. 381 further involves randomly selecting 562 any (second) piece of experimental content, and repeating 564 processes 552-560 for this next (second) piece of experimental content. The processes of blocks 550-564 are repeated 566…” [0446], “The embodiment shown in FIG. 39A is directed to algorithm that assigns content to time-slot samples where the individual pieces of content are shorter than the time-slot samples. The algorithm of FIG. 39A ensures that there are no content confounds and allows the same time-slot samples to be used to test multiple hypotheses (i.e., allows unrelated independent variables to be tested within the same time-slot samples). This is analogous to being able to test multiple drugs on the same patients, which saves time and money. For example, in a drug testing scenario, one can test a topical analgesic cream on the same patient who is being used to test a halitosis cure. That is, the topical analgesic cream should not impact halitosis and the halitosis cure should not impact a skin condition. However, one would not want to test a treatment for halitosis on the same patients who are being used for testing a new toothpaste, for example.” [0451]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple combination by the features of Brooks, and in particular, to include in the test deployment and data collection interface configuration of Glover, in the Glover/Apple combination, the feature of ensuring that the subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests, as taught by Brooks, because it would help to “increase[s] the speed of experimentation as it is possible to simultaneously conduct experiments addressing multiple business objectives, thus liberating display time to achieve business goals.” (Brooks, [0434]).
As to Claim 2, the Glover/Apple/Brooks combination discloses as discussed above. Glover further discloses a streaming data ingestion engine (database 130) configured to receive the test data from the analytics server (Fig.2A, [0098]).
As to Claim 6, the Glover/Apple/Brooks combination discloses as discussed above. Glover further discloses wherein the test data comprises web traffic associated with a test deployment (“click-through rate, is monitored when users are tested using a plurality of test promotions,” [0158]) the test deployment defining a test group and a control group (“control variables,” [0087]).
As to Claim 7, the Glover/Apple/Brooks combination discloses as discussed above. Glover further discloses wherein the plurality of test types includes one or more of: a pre/post test, an event test, a session test, a visitor test, a known user tests, a targeted audience test (“The groups of consumers involved in promotion testing represent segments of the public that have been purposefully segmented” [0059], [0062]), or a geographic region-specific test ([0089]).
As to Claim 8, the Glover/Apple/Brooks combination discloses as discussed above. Glover further discloses wherein the test definition environment is communicatively connected to a retail environment including an online retail environment and a point of sale environment (“the point of sale terminal in a store, a shopping basket on an online shopping website, an app on a smart phone, a webpage displayed on a computer, a social media news feed , etc. where user responses can be received” [0117]-[0118]), the test definition environment configured to deploy a test to at least one of the online retail environment or the point of sale environment and receive traffic information from an analytics server responsive to the test (“click-through rate, is monitored when users are tested using a plurality of test promotions,” [0158], “the user response is measured by the click-through rate (CTR).” [0159]).
As to Claim 9, the Glover/Apple/Brooks combination discloses as discussed above. Glover further discloses wherein the test definition environment manages a test deployment across a plurality of test deployment stages (see stages depicted in Fig.2B and “steps” [0102]).
As to Claim 26, the Glover/Apple/Brooks combination discloses as discussed above. Glover further discloses wherein the plurality of tests includes a third test having a third set of characteristics, wherein the third set of characteristics includes at least one characteristic that is determined to be non-orthogonal to at least one characteristic of the first and second tests such that the third test is not executed by the test deployment and data collection interface simultaneously to the first and second test and is deferred in the management phase until completion of the first test and second test (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “…promotion testing may be iterated over and over with different subpopulations (segmented using the same or different segmenting criteria) and different test promotions (devised using the same or different combinations of test promotion variables) in order to validate one or more the test promotion response analysis result(s) prior to the formation of the generalized public promotion” [0066], “test promotions may be executed iteratively and/or in a continual fashion on different purposefully segmented subpopulations using different combinations of test promotion variables to continue to obtain insights into consumer actual revealed preferences, even as those preferences change over time.” [0071], “In one or more embodiments, once a set of concepts is developed, the technology advantageously a) will constrain offers to only test ‘viable promotions’, i.e., those that don’t violate local laws, conflict with branding guidelines, lead to unprofitable concepts that wouldn't be practically relevant, can be executed on a retailers' system, etc., and/or b) link to the design of experiments for micro-testing to determine which combinations of variables to test at any given time” [0080], “In an embodiment, it is envisioned that dozens, hundreds, or even thousands of these test promotions may be administered concurrently or staggered in time to the dozens, hundreds or thousands of segmented subpopulations. Further, the large number of test promotions executed (or iteratively executed) improves the statistical validity of the correlations ascertained by analysis engine” [0101]).
Claims 3-5 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Glover in view of Apple, in view of Brooks, and further in view of Klimetschek et al. (US 2015/0100406 A1)(“Klimetschek”).
As to Claim 3, the Glover/Apple/Brooks combination discloses as discussed above.
Glover does not directly disclose but Klimetschek teaches a metadata enrichment subsystem (“an Application Programming Interface (API) [0126]) receiving the test data from a web traffic ingestion engine (“The analytics service device 20 shown in FIG.1 can be used to collect analytics data 23 received via network 5” [0061]) and applying metadata descriptive of one or more attributes of the test data (“call a method or add an attribute in the source code of the component and/or to add metadata to component to describe what properties and events can be tracked,” [0126]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple/Brooks combination by the feature of Klimetschek and in particular to include in the Glover/Apple/Brooks combination, the feature of a metadata enrichment subsystem receiving the test data from a web traffic ingestion engine and applying metadata descriptive of one or more attributes of the test data, as taught by Klimetschek.
A person having ordinary skill in the art would have been motivated to combine these features “to expose tracking data for consumption within the system or systems that incorporate the component” (Klimetschek, [0084]) thereby facilitating the processing of the data.
As to Claim 4, the Glover/Apple/Brooks/Klimetschek combination discloses as discussed above. Glover does not directly disclose but Klimetschek teaches a metrics processing engine configured to generate a plurality of metrics based on enriched test data (“an analytics service running on an analytics service device 20. Instrumenting the component allows the component to expose tracking data for consumption within the system or systems that incorporate the component (or instances 308b, 308c, 308c'). Data may be exposed in a uniform way so that any other internal subsystem may readily access it. For example, this may be achieved by sending tracking data to JavaScript key-value stores, grouped under a particular subsystem that resides in a browser (see, e.g., content player 33 in FIG. 1) and stores data” [0084], “Statistics and metrics for all visitors can be displayed in place, and aggregated into different segments or audiences of visitors.” [0085]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple/Brooks/Klimetschek combination by the feature of Klimetschek and in particular to include in the Glover/Apple/Brooks/Klimetschek combination, the feature of a metrics processing engine configured to generate a plurality of metrics based on enriched test data, as taught by Klimetschek.
A person having ordinary skill in the art would have been motivated to combine these features because it would help to improve the accuracy of the analysis of the test data.
As to Claim 5, the Glover/Apple/Brooks/Klimetschek combination discloses as discussed above. Glover does not directly disclose but Klimetschek teaches a data storage system configured to store enriched test data received from the metadata enrichment subsystem and metrics generated by the metrics processing engine (“sending tracking data to JavaScript key-value stores, grouped under a particular subsystem that resides in a browser (see, e.g., content player 33 in FIG. 1) and stores data” [0084]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple/Brooks/Klimetschek combination by the feature of Klimetschek and in particular to include in the Glover/Apple/Brooks/Klimetschek combination, the feature of a data storage system configured to store enriched test data received from the metadata enrichment subsystem and metrics generated by the metrics processing engine, as taught by Klimetschek.
A person having ordinary skill in the art would have been motivated to combine these features because it would help to improve the accuracy of the analysis of the test data.
As to Claim 10, the Glover/Apple/Brooks combination discloses as discussed above.
Glover does not directly disclose but Klimetschek teaches wherein the plurality of test deployment stages include a proposed test stage (editing/authoring stage, [0108]-[0111]), an approved test stage (test and target stage, [0113]), a live test stage (live stage, [0114]), a completed test stage (auto mode, [0116]), and an archived test stage ([0112]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple/Brooks combination by the feature of Klimetschek and in particular to include in the Glover/Apple/Brooks combination, the feature of wherein the plurality of test deployment stages include a proposed test stage, an approved test stage, a live test stage, a completed test stage, and an archived test stage, as taught by Klimetschek.
A person having ordinary skill in the art would have been motivated to combine these features because it would help a user to better manage and control the testing.
As to Claim 11, the Glover/Apple/Brooks combination discloses as discussed above.
Glover does not directly disclose but Klimetschek teaches wherein the analysis interface (analytics UI 402) graphically displays test results associated with one or more selected channels of the plurality of content delivery channels (“The analytics information can include statistics and metrics gleaned from the analytics data 23 collected by the analytics service device 20. Statistics and metrics for all visitors can be displayed in place, and aggregated into different segments or audiences of visitors” [0085], “The analytics UI 402 also displays graphs 426b-d of metrics and statistics for instances 420b-d (i.e., different experiences).” [0086]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple/Brooks combination by the feature of Klimetschek and in particular to include in the Glover/Apple/Brooks combination, the feature of wherein the analysis interface graphically displays test results associated with one or more selected channels of the plurality of content delivery channels, as taught by Klimetschek.
A person having ordinary skill in the art would have been motivated to combine these features because it would help a user to better manage and control the testing.
As to Claim 12, the Glover/Apple/Brooks/Klimetschek combination discloses as discussed above.
Glover does not directly disclose but Apple teaches wherein the one or more selected channels includes at least one channel that is different from a channel associated with the test deployment (“In another example, the independent variable may include different Media Device 0110, Media Program, or Media Location through which Advertiser System 0100 transmits an Advertisement.” [0936], “The present invention can determine whether an Advertisement delivered to any Media Device 0110 results in a sale in any Retail Channel,” [0706], “By importing data on the retail channel in which a customer bought a product, the report can enable an Advertiser/Manufacturer to measure the ROI of an Advertisement viewed through any Media Device 0110, regardless of the retail channel.” [1150]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple/Brooks/Klimetschek combination by the feature of Apple and in particular to include in the Glover/Apple/Brooks/Klimetschek combination, the feature of wherein the one or more selected channels includes at least one channel that is different from a channel associated with the test deployment, as taught by Apple.
A person having ordinary skill in the art would have been motivated to combine these features “so that the online promotion testing result can more accurately forecast the actual in-store consumer behavior” (Glover, [0155]).
Response to Arguments
Applicant’s arguments filed in the August 2025 Remarks have been fully considered and addressed below.
On pages 9-10, Applicant argues that “neither Glover, Apple, nor Klimetschek disclose the active selection and coordination of orthogonal tests, nor are they alleged to” and “First, the cited portions of Brooks simply disclose simultaneously testing different variables during the same time slot. Brooks does not disclose or suggest a system for simultaneously deploying tests, each test being defined by a set of characteristics. Furthermore, while Brooks mentions that variables can be simultaneously tested during the same time slot, Brooks does not even provide a system which selects such variables from among a plurality of variables in a management phase. As such, Brooks does not, nor can it, disclose a system configured to select a subset of tests for simultaneous execution by determining whether the set of characteristics of a first set are orthogonal to the set of characteristics of a second test, such that the two tests are orthogonal to one another and can simultaneously advance to an execution phase.” In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). In the rejection, Glover is relied upon, inter alia, for the disclosure of:
wherein the plurality of tests are defined within the test definition retail environment as test proposals having a defined set of characteristics including a test type (“manner of discount” [0061]), a definition of a user population (“subpopulation,” [0059]), a duration (“date,” [0076]), a proposed change (“change,” [0092]), and a selection of one or more channels for test deployment from among the plurality of digital and non-digital purchase channels (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “Generally speaking, each individual test promotion may be designed to test one or more test promotion variables. These test promotions variables may relate to, for example, the size, shape, color, manner of display, manner of discount, manner of publicizing, manner of dissemination pertaining to the goods/services being promoted” [0061]),
wherein the test deployment and data collection interface is configured to select a subset of the plurality of tests for simultaneous execution (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “…promotion testing may be iterated over and over with different subpopulations (segmented using the same or different segmenting criteria) and different test promotions (devised using the same or different combinations of test promotion variables) in order to validate one or more the test promotion response analysis result(s) prior to the formation of the generalized public promotion” [0066], “test promotions may be executed iteratively and/or in a continual fashion on different purposefully segmented subpopulations using different combinations of test promotion variables to continue to obtain insights into consumer actual revealed preferences, even as those preferences change over time.” [0071], “In one or more embodiments, once a set of concepts is developed, the technology advantageously a) will constrain offers to only test ‘viable promotions’, i.e., those that don't violate local laws, conflict with branding guidelines, lead to unprofitable concepts that wouldn't be practically relevant, can be executed on a retailers' system, etc., and/or b) link to the design of experiments for micro-testing to determine which combinations of variables to test at any given time” [0080], “In an embodiment, it is envisioned that dozens, hundreds, or even thousands of these test promotions may be administered concurrently or staggered in time to the dozens, hundreds or thousands of segmented subpopulations. Further, the large number of test promotions executed (or iteratively executed) improves the statistical validity of the correlations ascertained by analysis engine” [0101]) in a management phase (“promotional idea module,” [0079]),
wherein the plurality of tests includes a first test having a first set of characteristics and a second test having a second set of characteristics (“One or more of the test promotion variables may vary from test promotion to test promotion. In the example of FIG. 2A, test promotion 102a involves test variable X1 (representing a given value or attribute for test variable X) while test promotion 102b involves test variable X2 (representing a different value or attribute for test variable X). A test promotion may vary, relative to another test promotion, one test promotion variable (as can be seen in the comparison between test promotions 102a and 102b) or many of the test promotion variables (as can be seen in the comparison between test promotions 102a and 102d).” [0087]),
wherein the first set of characteristics and second set of characteristics are determined to be orthogonal to one another such that the first test and second test are simultaneously executed (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “…promotion testing may be iterated over and over with different subpopulations (segmented using the same or different segmenting criteria) and different test promotions (devised using the same or different combinations of test promotion variables) in order to validate one or more the test promotion response analysis result(s) prior to the formation of the generalized public promotion” [0066], “test promotions may be executed iteratively and/or in a continual fashion on different purposefully segmented subpopulations using different combinations of test promotion variables to continue to obtain insights into consumer actual revealed preferences, even as those preferences change over time.” [0071], “In one or more embodiments, once a set of concepts is developed, the technology advantageously a) will constrain offers to only test ‘viable promotions’, i.e., those that don't violate local laws, conflict with branding guidelines, lead to unprofitable concepts that wouldn't be practically relevant, can be executed on a retailers' system, etc., and/or b) link to the design of experiments for micro-testing to determine which combinations of variables to test at any given time” [0080], “In an embodiment, it is envisioned that dozens, hundreds, or even thousands of these test promotions may be administered concurrently or staggered in time to the dozens, hundreds or thousands of segmented subpopulations. Further, the large number of test promotions executed (or iteratively executed) improves the statistical validity of the correlations ascertained by analysis engine” [0101]) by the test deployment and data collection interface and advance to an execution phase (“the test promotions may be generated using automated test promotion generation software 110, which varies for example the test promotion variables and/or the values of the test promotion variables and/or the number of the test promotion variables to come up with different test promotions.” [0088]).
The rejection further states that Glover does not directly disclose:
to ensure that the subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests.
The rejection further states that Brooks teaches ensuring that a subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests (“The methodology disclosed in this application also allows the ability to simultaneously test multiple independent variables during the same time-slot samples, providing that the content associated with the different independent variables is unrelated. This is because experimental content for one independent variable can be control content for another independent variable.” [0434], “The algorithm of FIG. 381 further involves randomly selecting 562 any (second) piece of experimental content, and repeating 564 processes 552-560 for this next (second) piece of experimental content. The processes of blocks 550-564 are repeated 566…” [0446], “The embodiment shown in FIG. 39A is directed to algorithm that assigns content to time-slot samples where the individual pieces of content are shorter than the time-slot samples. The algorithm of FIG. 39A ensures that there are no content confounds and allows the same time-slot samples to be used to test multiple hypotheses (i.e., allows unrelated independent variables to be tested within the same time-slot samples). This is analogous to being able to test multiple drugs on the same patients, which saves time and money. For example, in a drug testing scenario, one can test a topical analgesic cream on the same patient who is being used to test a halitosis cure. That is, the topical analgesic cream should not impact halitosis and the halitosis cure should not impact a skin condition. However, one would not want to test a treatment for halitosis on the same patients who are being used for testing a new toothpaste, for example.” [0451]).
The rejection further states that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Glover/Apple combination by the features of Brooks, and in particular, to include in the test deployment and data collection interface configuration of Glover, in the Glover/Apple combination, the feature of ensuring that the subset of tests for simultaneous execution are orthogonal to one another to prevent interference between the tests, as taught by Brooks, because it would help to “increase[s] the speed of experimentation as it is possible to simultaneously conduct experiments addressing multiple business objectives, thus liberating display time to achieve business goals.” (Brooks, [0434]).
As such, the argument is found unpersuasive.
On pages 10-12, Applicant argues the newly added Claim 26. However, as stated in the rejection: Glover further discloses wherein the plurality of tests includes a third test having a third set of characteristics, wherein the third set of characteristics includes at least one characteristic that is determined to be non-orthogonal to at least one characteristic of the first and second tests such that the third test is not executed by the test deployment and data collection interface simultaneously to the first and second test and is deferred in the management phase until completion of the first test and second test (“multiple test promotions may be executed on different subpopulations, either simultaneously or at different times” [0060], “…promotion testing may be iterated over and over with different subpopulations (segmented using the same or different segmenting criteria) and different test promotions (devised using the same or different combinations of test promotion variables) in order to validate one or more the test promotion response analysis result(s) prior to the formation of the generalized public promotion” [0066], “test promotions may be executed iteratively and/or in a continual fashion on different purposefully segmented subpopulations using different combinations of test promotion variables to continue to obtain insights into consumer actual revealed preferences, even as those preferences change over time.” [0071], “In one or more embodiments, once a set of concepts is developed, the technology advantageously a) will constrain offers to only test ‘viable promotions’, i.e., those that don’t violate local laws, conflict with branding guidelines, lead to unprofitable concepts that wouldn't be practically relevant, can be executed on a retailers' system, etc., and/or b) link to the design of experiments for micro-testing to determine which combinations of variables to test at any given time” [0080], “In an embodiment, it is envisioned that dozens, hundreds, or even thousands of these test promotions may be administered concurrently or staggered in time to the dozens, hundreds or thousands of segmented subpopulations. Further, the large number of test promotions executed (or iteratively executed) improves the statistical validity of the correlations ascertained by analysis engine” [0101]).
Therefore, the argument is found unpersuasive.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MONICA A MANDEL whose telephone number is (571)270-7046. The examiner can normally be reached Monday and Thursday 10:00 AM-6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ilana Spar can be reached at (571) 270-7537. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.A.M/Examiner, Art Unit 3622
/ILANA L SPAR/Supervisory Patent Examiner, Art Unit 3622