DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Application
This office action is in response to the most recent response filed by applicants on 11/26/25.
Claims 1, 9, 14, 16 and 22 are amended
Claims 4, 6 and 11 are cancelled
No claims are added
Claims 1-3, 5, 7-10, 12-23 are pending
Note:
In the amended independent claims 1, 9 and 16, applicants have added the amended claim limitation “determining an adjusted attribution for the event based on the direct attribution for the event augmented with an indirect attribution for the event, the indirect attribution identified, via the machine learned conditional intensity model, based on the event causing a subsequent event of the set of events to result in the conversion and the indirect attribution for the event, wherein a causal graph is generated based on causality coefficients determined during training of the machine learned conditional intensity model and includes a set of nodes representing the set of events and a set of directed edges representing a causal influence between connected events of the set of events and the indirect attribution for the event is determined by at least backpropagating attribute credit associated with the at least the subsequent event to the event along directed edges of the set of directed edges;”
The specification shows the above limitations at least in [0018]: In embodiments, a conditional intensity model is used to determine attribution for events. A conditional intensity model refers to a spatial point process describing how the probability that an event in the process occurs at a particular point depends on the prior point events in the process. In implementation, in determining conditional intensities, a baseline parameter and a causal parameter (e.g., Granger causality parameter) are used. A baseline parameter generally indicates an extent of propensity for an event to occur without any stimulus. A causal parameter generally indicates an extent of excitation on an occurrence of another event. Such parameters are learned during training of the conditional intensity model. In this regard, the conditional intensity model is trained to fit the baseline parameter and the causal parameter to result in a function that measures the propensity of an event occurring in a certain time span. To efficiently train the conditional intensity model, hyperparameters, such as sigma, can be determined such that they are set in advance of learning the baseline and causal parameters. Stated differently, to facilitate learning of baseline parameters and/or causal parameters, hyperparameters are established in advance of training the conditional intensity model. In particular, the machine learning algorithms can include preset hyperparameters, including the sigma parameters. Such a sigma hyperparameter determines or specifies the rate of decay function.
Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Here, the details in the specification are missing from the claim. Regarding 101, the claim is recited at a very high level of generality. Similarly, for the 103 rejections, the previously presented prior art still reads on the claim.
In light of these notes, the amended claims, do not overcome previously presented rejections under 101 and 103. As is discussed below. This note is intended as a conversation starter to help applicants understand the examiner’s perspective. Applicants are welcome to call the examiner to discuss this further.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 5, 7-10, 12-23 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more.
Step One - First, pursuant to step 1 in the January 2019 Guidance on 84 Fed. Reg. 53, the claims 1-3, 5, 7-8, and 21-23 is/are directed to a method which is a statutory category.
Step One - First, pursuant to step 1 in the January 2019 Guidance on 84 Fed. Reg. 53, the claims 9-10, 12-15 is/are directed to a computer readable medium which is a statutory category.
Step One - First, pursuant to step 1 in the January 2019 Guidance on 84 Fed. Reg. 53, the claims 16-20 is/are directed to a system which is a statutory category.
Under the 2019 PEG, Step 2A under which a claim is not “directed to” a judicial exception unless the claim satisfies a two-prong inquiry. Further, particular groupings of abstract ideas are consistent with judicial precedent and are based on an extraction and synthesis of the key concepts identified by the courts as being abstract.
With respect to the Step 2A, Prong One, in independent claims 1 and 9, the claims as drafted, and given their broadest reasonable interpretation, fall within the Abstract idea grouping of “certain methods of organizing human activity” (business relations; relationships or interactions between people). For instance, Independent Claim 1 is directed to an abstract idea, as evidenced by claim limitations “obtaining a set of events comprising touchpoints resulting in a conversion; determining, a direct attribution indicating credit for an event, of the set of events, contributing to the conversion; determining an adjusted attribution for the event based on the direct attribution for the event augmented with an indirect attribution for the event, the indirect attribution identified, based on the event causing a subsequent event of the set of events to result in the conversion and the indirect attribution for the event, wherein a causal graph is generated based on causality coefficients determined during training and includes a set of nodes representing the set of events and a set of directed edges representing a causal influence between connected events of the set of events and the indirect attribution for the event is determined by at least backpropagating attribute credit associated with the at least the subsequent event to the event along directed edges of the set of directed edges; and providing the adjusted attribution for the event to indicate an extent of credit assigned to the event for causing the conversion.”
Similarly, independent claim 16, “determining a hyperparameter associated with a decay rate for use, the hyperparameter being determined by fitting a function to a distribution of a time difference of two events of a pair of event types, wherein the two events are from a same event path; and training using the determined hyperparameter, to identify a causal parameter indicating an extent of excitation of an occurrence of another event; and generating a causal graph based on the causal parameter, the causal graph including a set of nodes representing event types and a set of directed edges between nodes of the set of nodes representing causal influence between event types, wherein the conditional intensity model and the causal graph are used to identify direct attribution and indirect attribution for an event.”
These claim limitations belong to the grouping of “certain methods of organizing human activity” because the claims are related to facilitating generation and utilization of causal-based attribution models. Providing an efficient causal-based attribution model used to determine attributions for various events, or touchpoints, of event paths leading to conversions. At a high level, using a causal-based attribution model enables back propagation of attribution credit to earlier events or touchpoints that cause later events or touchpoints, thereby capturing causal relationships between different marketing touchpoints. Advantageously, using a causal-based attribution model enables a more accurate estimation of contribution from various events. In particular, redistributing attribution credit from later touchpoints to earlier touchpoints, in cases in which the earlier touchpoints were found to cause later touchpoints, provides the earlier touchpoints that drive other touchpoints with higher credit. (See specification [0002]- [0003].).
Managing the ability to attribute the cause of a particular behavior/event to the behavior/event for one or more human entities involves organizing human activity based on the description of “certain methods of organizing human activity” provided by the courts. For instance, if a coupon is being used by a customer to buy a particular product, attributing the increased sales of the product to the coupon issued allows a company to make better decisions on whether issuing the coupon is an effective marketing tool for the sale of a particular product or service. The courts have used the phrase “Certain methods of organizing human activity” as —fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions).
Independent Claim 9 is/are recite substantially similar limitations to independent claims 1 and 16 and is/are rejected under 2A for similar reasons to claims 1 and 16 above.
With respect to the Step 2A, Prong Two - This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements: “A method comprising: via a machine learned conditional intensity model, One or more computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method comprising: (claim 1). A computing system comprising: in training a machine learning conditional intensity model (claim 16)” at a high level of generality such that it amounts to no more than: adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f). Thus, the additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limitations on practicing the abstract idea. As a result, claims 1, 9 and 16 do not provide any specifics regarding the integration into a practical application when recited in a claim with a judicial exception. See MPEP 2106.05(f).
Similarly dependent claims 2-3, 5, 7-8, 10, 12-15 and 17-23 are also directed to an abstract idea under 2A, first and second prong. In the present application, all of the dependent claims have been evaluated and it was found that they all inherit the deficiencies set forth with respect to the independent claims. For instance, dependent claims 2 recite “further comprising: determining a hyperparameter associated with a decay rate for use in training the machine learned conditional intensity model, the hyperparameter being determined by fitting an exponential to a distribution of a time difference of two events of a pair of event types, wherein the two events are from a same event path”. Here, these claims offer further descriptive limitations of elements found in the independent claims which are similar to the abstract idea noted in the independent claim above.
Dependent claim 5 recites “generating the casual graph” in the claim limitations “further comprising generating the causal graph, wherein the causal graph is generated based on causal parameters learned in association with training the machine learned conditional intensity model.” In this claim, “generating the casual graph” is an additional element, but it is still being recited such that it amounts to no more than: adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f). As a result, Examiner asserts that dependent claims, such as dependent claims 2-3, 5, 7-8, 10, 12-15 and 17-23 are also directed to the abstract idea identified above.
With respect to Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. First, the invention lacks improvements to another technology or technical field [see Alice at 2351; 2019 IEG at 55], and lacks meaningful limitations beyond generally linking the use of an abstract idea to a particular technological environment [Alice at 2360, 2019 IEG at 55], and fails to effect a transformation or reduction of a particular article to a different state or thing [2019 IEG, 55]. For the reasons articulated above, the claims recite an abstract idea that is limited to a particular field of endeavor (MPEP § 2106.05(h)) and recites insignificant extra-solution activity (MPEP § 2106.05(g)). By the factors and rationale provided above with respect to these MPEP sections, the additional elements of the claims that fail to integrate the abstract idea into a practical application also fail to amount to “significantly more” than the abstract idea.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of “A method comprising: via a machine learned conditional intensity model, One or more computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:” (claim 1) and “A computing system comprising: in training a machine learning conditional intensity model” (claim 16) are insufficient to amount to significantly more. Applicants originally submitted specification describes the computer components above at least in page/ paragraph [0022]-[0029], [0036]-[0039], [0086], [0090]-[0092]. In light of the specification, it should be noted that the components discussed above did not meaningfully limit the abstract idea because they merely linked the use of the abstract idea to a particular technological environment (i.e., "implementation via computers"). In light of the specification, it should be noted that the claim limitations discussed above are merely instructions to implement the abstract idea on a computer. See MPEP 2106.05(f). (See MPEP 2106.05(f) - Mere Instructions to Apply an Exception - “Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible.” Alice Corp., 134 S. Ct. at 235). Mere instructions to apply an exception using computer component cannot provide an inventive concept.). The additional elements amount to no more than a recitation of generic computer elements utilized to perform generic computer functions, such as performing repetitive calculations, Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012) ("The computer required by some of Bancorp’s claims is employed only for its most basic function, the performance of repetitive calculations, and as such does not impose meaningful limits on the scope of those claims."); and storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; see MPEP 2106.05(d)(II).
The claim fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine the claim to a particular useful application, and/or meaningful limitations beyond generally linking the use of an abstract idea to a particular environment. See 84 Fed. Reg. 55. Viewed individually or as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself.
Independent Claims 9 is/are recite substantially similar limitations to independent claims 1 and 16 and is/are rejected under 2B for similar reasons to claims 1 and 16 above.
Further, it should be noted that additional elements of the claimed invention such as claim limitations when considered individually or as an ordered combination along with the other limitations discussed above in method claim 1 also do not meaningfully limit the abstract idea because they merely linked the use of the abstract idea to a particular technological environment (i.e., "implementation via computers"). In light of the specification, it should be noted that the claim limitations discussed above are merely instructions to implement the abstract idea on a computer. See MPEP 2106.
Similarly, dependent claims 2-3, 5, 7-8, 10, 12-15 and 17-23 also do not include limitations amounting to significantly more than the abstract idea under the second prong or 2B of the Alice framework. In the present application, all of the dependent claims have been evaluated and it was found that they all inherit the deficiencies set forth with respect to the independent claims. Further, it should be noted that the dependent claims do not include limitations that overcome the stated assertions. Here, the dependent claims recite features/limitations that include computer components identified above in part 2B of analysis of independent claims 1, 9 and 16. As a result, Examiner asserts that dependent claims, such as dependent claims 2-3, 5, 7-8, 10, 12-15 and 17-23 are also directed to the abstract idea identified above.
For more information on 101 rejections, see MPEP 2106, January 2019 Guidance at https://www.govinfo.gov/content/pkg/FR-2019-01 -07/pdf/2018-28282.pdf
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 7-10, 12-15 and 21-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhong et al. (2018/0308123), further in view of Curcio et al. (2018/0232264) and Councill et al. (US 2023/0367782).
As per claims 1 and 9: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
A method comprising:
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
obtaining a set of events comprising touch points resulting in a conversion (Zhong shows: [0010] In this disclosure, the term “touch point” (also referred to as touchpoint, contact point, or point of contact) refers to any encounter between a consumer and a business. For example, a listener heard an ad about a business on the radio. In this case, the radio represents an offline channel and the ad represents an offline advertising event occurring via the offline channel. Suppose this is the first time the listener encountered the business. This encounter represents an offline touch point. Suppose the listener then went online and visited a website of the business and, while there, made a purchase through the website. The listener's visit to the business's website represents an online touch point. Those skilled in the art will recognize that, whether it is offline or online, a channel can have numerous touch points. [0011] Although it appears that the online touch point resulted in a conversion—the listener made a purchase through the website, in this example, the offline touch point is what caused the listener to visit the business's website in the first place. To be fair and accurate, then, the offline advertising event deserves some credit for the conversion—in other words, this particular conversion should ideally be fractionally attributed to the offline advertising event occurring in the offline channel. However, as the above example illustrates, offline touch points and online touch points may not overlap. Therefore, it can be very difficult, if not impossible, to combine and/or properly associate offline touch points with online touch points.);
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“determining, via a machine learned conditional intensity model, a direct attribution indicating credit for an event, of the set of events, contributing to the conversion”
Applicants’ specification shows the limitation “machine learned conditional intensity model” in [0003]: a conditional intensity model is used to determine attribution for events. In implementation, in determining conditional intensities, a baseline parameter and a causal parameter (e.g., Granger causality parameter) are used. Such parameters are learned during training of the conditional intensity model. In this regard, the conditional intensity model is trained to fit the baseline parameter and the causal parameter to result in a function that measures the propensity of an event occurring in a certain time span. To efficiently train the conditional intensity model, hyperparameters can be determined such that it is set in advance of learning the baseline and causal parameters. Stated differently, to facilitate learning of baseline parameters and/or causal parameters, hyperparameters, such as a sigma (or -) parameter, is established in advance of training the conditional intensity model. In accordance with embodiments described herein, the predetermined hyperparameters are determined in an efficient and scalable manner prior to performing machine learning to learn model parameters.
In light of this description, Zhong shows [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. Zhong: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0083]-[0089]: credit conversion and fractional credit attribution.
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“determining an adjusted attribution for the event based on the direct attribution for the event augmented with an indirect attribution for the event, the indirect attribution identified, via the machine learned conditional intensity model, based on the event causing a subsequent event of the set of events to result in the conversion and the indirect attribution for the event, wherein a causal graph is generated based on causality coefficients determined during training of the machine learned conditional intensity model and includes a set of nodes representing the set of events and a set of directed edges representing a causal influence between connected events of the set of events and the indirect attribution for the event is determined by at least backpropagating attribute credit associated with the at least the subsequent event to the event along directed edges of the set of directed edges”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n. This reads on “determining an adjusted attribution for the event based on the direct attribution for the event augmented with an indirect attribution for the event”
Zhong also shows “event augmented” in the claim above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight
Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0083]-[0089]: credit conversion and fractional credit attribution “the indirect attribution identified, via the machine learned conditional intensity model, based on the event causing a subsequent event of the set of events to result in the conversion, wherein a causal graph is generated based on causality coefficients determined during training of the machine learned conditional intensity model and includes a set of nodes representing the set of events and a set of directed edges representing a causal influence between connected events of the set of events and the indirect attribution for the event is determined by at least backpropagating attribute credit associated with the at least the subsequent event to the event along directed edges of the set of directed edges”.
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“wherein a causal graph is generated based on causality coefficients determined during training of the machine learned conditional intensity model and includes a set of nodes representing the set of events and a set of directed edges representing a causal influence between connected events of the set of events and the indirect attribution for the event is determined by at least backpropagating attribute credit associated with the at least the subsequent event to the event along directed edges of the set of directed edges.”
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does). Zhong shows: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
However, Zhong does not explicitly show the above limitation. Curcio shows the above limitation at least in [0020]: A system 100 includes components for (i) storing user interactions with each of the various web sites and/or an advertisement on the various web sites, (ii) attributing conversion events to the user interactions using a browsing history of each user, and (iii) aggregating data regarding the conversion events to facilitate reporting. [0042] Referring back to FIG. 1, the server 110 may also include an aggregation engine 140 that receives the attributed conversions 145 from the attribution engine 120. The aggregation engine 140 may aggregate the data and transmit the data to a reporting engine 160. The reporting engine 160 may be configured to output the data to an end user for analysis.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A).
Further, Neither Zhong nor Curcio show “coefficients” discussed in the claim above. However, Councill shows the above limitation at least in [0012] In at least one embodiment, one or more of the steps to identify the representative set of data for each source database, to generate granular data types for at least a portion of the data of the meta-database, to determine the plurality of variables indicative of at least a portion of the data of the meta-database, or to produce at least one association between at least two variables may be included in instructions associated with a machine learning algorithm. In an additional or alternative embodiment, the machine learning algorithm may use a neural network. In a further or alternative embodiment, the instruction to generate a probability distribution for each variable of the plurality of variables may include instructions to apply a function of the granular data type for at least one variable, the function including at least one of a density estimate, discrete distribution, or sample. Additionally or alternatively, the instruction to produce at least one association between at least two variables may include instructions to determine or estimate at least one of a parametric correlation between the at least two variables, a non-parametric correlation between the at least two variables, a Pearson correlation between the at least two variables, a Spearman correlation between the at least two variables, a Kendall's Tau correlation between the at least two variables, mutual information between the at least two variables, an uncertainty coefficient between the at least two variables, or a causal relationship between the at least two variables. [0073] Neural networks may perform a supervised learning process where known inputs and known outputs are utilized to categorize, classify, or predict a quality of a future input. However, additional or alternative embodiments of the machine learning program may be trained utilizing unsupervised or semi-supervised training, where none of the outputs or some of the outputs are unknown, respectively. Typically, a machine learning algorithm is trained (e.g., utilizing a training data set) prior to modeling the problem with which the algorithm is associated. Supervised training of the neural network may include choosing a network topology suitable for the problem being modeled by the network and providing a set of training data representative of the problem. Generally, the machine learning algorithm may adjust the weight coefficients until any error in the output data generated by the algorithm is less than a predetermined, acceptable level. For instance, the training process may include comparing the generated output produced by the network in response to the training data with a desired or correct output. An associated error amount may then be determined for the generated output data, such as for each output data point generated in the output layer. The associated error amount may be communicated back through the system as an error signal, where the weight coefficients assigned in the hidden layer are adjusted based on the error signal. For instance, the associated error amount (e.g., a value between −1 and 1) may be used to modify the previous coefficient, e.g., a propagated value. The machine learning algorithm may be considered sufficiently trained when the associated error amount for the output data is less than the predetermined, acceptable level (e.g., each data point within the output layer includes an error amount less than the predetermined, acceptable level). Thus, the parameters determined from the training process can be utilized with new input data to categorize, classify, and/or predict other values based on the new input data. [0121] In some embodiments, an association produced between two or more variables (e.g., association 736 and/or association 738) represents or approximates a causal relationship between such variables. Such casual relationships may be determined between variables associated with a single source database 702 or data stored across multiple source databases 702. In some embodiments, determining such a causal relationship may include instructions associated with determining at least one parametric correlation between at least two variables, a non-parametric correlation between at least two variables, a Pearson correlation between at least two variables, a Spearman correlation between at least two variables, a Spearman correlation between at least two variables, a Kendall's Tau correlation between at least two variables, mutual information between at least two variables, or an uncertainty coefficient (e.g., a Theil's U coefficient) between at least two variables. In some embodiments, an AI program(s) of the profiler algorithm 717 may be used in the process of determining a causal relationship between variables. For example, a deep neural network may be trained to determine or approximate such a causal relationship and/or coefficients for determining the same, such as some or all of the proceeding. It should be appreciated that the AI program may be capable of approximating the causal relationship and/or coefficients or parameters representing the relationship or used to establish the relationship faster and/or require less computing power than the traditional mathematical operations to determine the same. Thus, the profiler algorithm 717 may expedite representing or approximating causal relationships between variables of the meta-database 708 in a pre-processing stage. Other AI programs that may be used, at least in part, by the profiler algorithm 717 to determine casual relationships between variables include machine learning programs, neural networks, CNNs, support vector algorithms, decision tree learning, associate rule learning, or, similar AI algorithms suitable for producing causal inferences based on input data.
Reference Zhong and Reference Councill are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Councill, particularly the ability to look at coefficients in the causality graph in [0012], [0073] and [0121], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system to generate granular data types for at least a portion of the data of the meta-database, to determine the plurality of variables indicative of at least a portion of the data of the meta-database, or to produce at least one association between at least two variables may be included in instructions associated with a machine learning algorithm as taught by Reference Councill (see at least in [0012], where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Councill, the results of the combination were predictable (MPEP 2143 A); and
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
providing the adjusted attribution for the event to indicate an extent of credit assigned to the event for causing the conversion
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
However, Zhong does not explicitly show the above limitation. Curcio shows the above limitation at least in [0020]: A system 100 includes components for (i) storing user interactions with each of the various web sites and/or an advertisement on the various web sites, (ii) attributing conversion events to the user interactions using a browsing history of each user, and (iii) aggregating data regarding the conversion events to facilitate reporting. [0042] Referring back to FIG. 1, the server 110 may also include an aggregation engine 140 that receives the attributed conversions 145 from the attribution engine 120. The aggregation engine 140 may aggregate the data and transmit the data to a reporting engine 160. The reporting engine 160 may be configured to output the data to an end user for analysis.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A).
As per claim 2:
Regarding the claim limitations below, Zhong in view of Curcio shows:
“A computing system comprising:
determining a hyperparameter associated with a decay rate for use in training a machine learning conditional intensity model, the hyperparameter being determined by fitting a function to a distribution of a time difference of two events of a pair of event types, wherein the two events are from a same event path”
Zhong shows: [0010] In this disclosure, the term “touch point” (also referred to as touchpoint, contact point, or point of contact) refers to any encounter between a consumer and a business. For example, a listener heard an ad about a business on the radio. In this case, the radio represents an offline channel and the ad represents an offline advertising event occurring via the offline channel. Suppose this is the first time the listener encountered the business. This encounter represents an offline touch point. Suppose the listener then went online and visited a website of the business and, while there, made a purchase through the website. The listener's visit to the business's website represents an online touch point. Those skilled in the art will recognize that, whether it is offline or online, a channel can have numerous touch points. [0011] Although it appears that the online touch point resulted in a conversion—the listener made a purchase through the website, in this example, the offline touch point is what caused the listener to visit the business's website in the first place. To be fair and accurate, then, the offline advertising event deserves some credit for the conversion—in other words, this particular conversion should ideally be fractionally attributed to the offline advertising event occurring in the offline channel. However, as the above example illustrates, offline touch points and online touch points may not overlap. Therefore, it can be very difficult, if not impossible, to combine and/or properly associate offline touch points with online touch points.
In light of the description in the spec [0003] “a machine learning conditional intensity model”, Zhong shows [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. Zhong: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0083]-[0089]: credit conversion and fractional credit attribution. Zhong shows “hyperparameter” at least in [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
Even though Zhong shows attribution weight for events and weight distribution to attribute credit correctly in [0083]-[0089], [0119], Zhong does not explicitly show “associated with a decay rate for use”. Curcio shows the above limitation at least in [0028] Generally, individual impressions (or clusters of impressions) in the user's history are analyzed by the attribution engine 120. The attribution model 120 determines which of the impressions (e.g., websites, advertisements) contributed to the conversion event 125. In certain embodiments, each user interaction with an advertisement on a web site is combined with a time function (e.g., a function of a length of time preceding the conversion event) to determine a weighted contribution or attribution value of each of the web sites in the browsing history. For example, a web site visited two days before the conversion event may not be given as much weight as a web site that was visited just one day, hours, or even minutes before the conversion event 125. In another embodiment, the aggregation engine 120 may be configured to determine a chain of web sites that contributed to the conversion event even though the web sites in the chain are not related in subject matter, either to each other, and/or to the subject matter of the conversion event 125. Time-Energy Calculations. [0032] The attribution engine 120 performs calculations on the incoming data. In one embodiment, the attribution engine 120 implements calculations that model each user as a system with a variable amount of energy that, over time, seeks a global baseline state. This “energy” is analogous to, and used as a proxy for, the user's presumed interest level. When viewed as an energy function, acts deemed to be of interest represent energy addition, subject to decay by time (as well as optionally subtraction by other intervening events). Different values in the energy curve indicate the different level of engagement that the user has. The user activities that correlate to the energy values comprise the events that receive attribution for a subsequent conversion event. [0033] More specifically, events in the history preceding the conversion event add or deplete energy from the user's interest level. In particular, one or more embodiments provide that certain events, such as, for example, click events, add energy to the model, while other events, such as, for example, tightly clustered ads, remove energy from the model. Energy injection and extraction for other reasons may be supported via additional types of event meta-data (e.g., ad size, placement, etc.). An attribution decay function is applied to further highlight the change of the model over time. This separate decay is a time effect representing the heuristic that temporally distant events from a conversion event should not be attributed credit for the conversion event or should be attributed less credit for the event. The attribution decay concept more accurately weights past events that contributed to a conversion event than the traditional attribution window. Under an embodiment described, for example, events preceding a conversion event are not modeled for attribution based solely on a timeline or preceding event, but rather analyzed over a preceding duration that takes into account events/activities that peak user interest. [0034] To calculate the energy of a chain, and thus the presumed interest level of a user, at a particular point in time, each event in a user chain is visited in order, from least to most recent. A prophetic example of a chain energy calculation is shown in FIG. 2A. The x-axis of FIG. 2A shows time before a conversion event: the conversion event occurs at time zero, and the farther along the x-axis an event is, the further back in time the event is. The y-axis of FIG. 2A is the modeled chain energy value, which is assumed to reflect the interest level of the user. If an event (e.g., an impression) is determined to be related to the conversion event, the chain energy value at the point in time when the event occurs is applied to the event as a weighting factor. In the example of FIG. 2A, the chain energy has a baseline value of 1.0, and increases (e.g., as a step function) to a value of 2.0 in response to a click event or other user interaction (e.g., in response to a user clicking on, or otherwise interacting with, an impression). After the click event (or other user interaction), the chain energy decreases toward the baseline value. For example, the chain energy decreases based on an exponential decay having a heuristically determined time constant. [0036] In some embodiments, when a conversion event occurs, an attribution decay function is combined with the chain energy function to arrive at the final attribution allotment. The attribution decay may be an exponential function that runs to zero at the end of the client-specified attribution window, as shown by FIG. 2B. The exponential function has a corresponding half-life, the value of which may be determined empirically and may vary from application to application. Examples of half-lives may be in the range of 1-5, or 1-10, or 1-30 or more. Alternatively, the attribution decay may have an alternative form, such as another time-dependent curve.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to account for a decay rate from the first touch point to the next [0028]-[0034], in the disclosure of Reference Zhong, particularly in the ability to attribute weight for events and weight distribution to attribute credit correctly in [0083]-[0089], [0119], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also account for loss of customer interest along the way as taught by Reference Curcio (see at least in [0028]-[0034]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A); and
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“training the machine learning conditional intensity model, using the determined hyperparameter, to identify a causal parameter indicating an extent of excitation of an occurrence of another event.”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n. Zhong shows in the above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight. Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved;
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“generating a causal graph based on the causal parameter, the causal graph including a set of nodes representing event types and a set of directed edges between nodes of the set of nodes representing causal influence between event types, wherein the conditional intensity model and the causal graph are used to identify direct attribution and indirect attribution for an event”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
Zhong also shows “event augmented” in the claim above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight
Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0083]-[0089]: credit conversion and fractional credit attribution.
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does). Zhong shows: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
However, Zhong does not explicitly show the above limitation. Curcio shows the above limitation at least in [0020]: A system 100 includes components for (i) storing user interactions with each of the various web sites and/or an advertisement on the various web sites, (ii) attributing conversion events to the user interactions using a browsing history of each user, and (iii) aggregating data regarding the conversion events to facilitate reporting. [0042] Referring back to FIG. 1, the server 110 may also include an aggregation engine 140 that receives the attributed conversions 145 from the attribution engine 120. The aggregation engine 140 may aggregate the data and transmit the data to a reporting engine 160. The reporting engine 160 may be configured to output the data to an end user for analysis.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A);
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“and generating a causal graph based on the causal parameter, the causal graph including a set of nodes representing event types and a set of directed edges between nodes of the set of nodes representing causal influence between event types, wherein the conditional intensity model and the causal graph are used to identify direct attribution and indirect attribution for an event.”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
Zhong also shows “event augmented” in the claim above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight
Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get.
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does). Zhong shows: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
However, Zhong does not explicitly show the above limitation. Curcio shows the above limitation at least in [0020]: A system 100 includes components for (i) storing user interactions with each of the various web sites and/or an advertisement on the various web sites, (ii) attributing conversion events to the user interactions using a browsing history of each user, and (iii) aggregating data regarding the conversion events to facilitate reporting. [0042] Referring back to FIG. 1, the server 110 may also include an aggregation engine 140 that receives the attributed conversions 145 from the attribution engine 120. The aggregation engine 140 may aggregate the data and transmit the data to a reporting engine 160. The reporting engine 160 may be configured to output the data to an end user for analysis.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A).
As per claims 3, 10 and 21: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the machine learned conditional intensity model is trained to fit data associated with event paths to learn model parameters including a baseline parameter and a causal parameter.
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n. Zhong shows in the above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight. Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved.
As per claims 5, 12 and 22: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
further comprising generating the causal graph, wherein the causal graph is generated based on causal parameters learned in association with training the machine learned conditional intensity model.
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
As per claims 13 and 23: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the indirect attribution for the event is determined based on a first causal parameter associated with the event and the subsequent event.
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
As per claims 7 and 15: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the direct attribution for the event is determined based on a difference of a first conditional intensity of conversion associated with the event and prior events at conversion time and a second conditional intensity of conversion associated with the prior events at the conversion time.
Zhang shows [0016]: a fractional attribution method may include arranging, by a computer, a plurality of channels into a plurality of funnel stages based on a conversion rate associated with each channel, constructing aggregate-level data, and computing a multi-stage regression on the plurality of funnel stages using the aggregate-level data to thereby determine channel weights for the plurality of channels. The order by which the plurality of funnel stages is arranged may be overridden based on domain knowledge. Where necessary, a channel may be split into sub-channels. A causal analysis may be performed on the plurality of channels using instrumental variables. Specifically, assume there are m levels in the plurality of funnel stages, a system implementing the causal analysis may first determine causal weights for the m.sup.th level channels using a two-stage least squares algorithm and using all channels above the m.sup.th level as instrumental variables. After the causal weights for the m.sup.th level channels are determined, the system may determine causal weights for m−1.sup.th level channels, with residual channels as dependent variables and channels above the m−1.sup.th levels as instrumental variables. This process may be repeated until all causal weights are determined for the plurality of channels.
Curcio shows the above limitations at least in [0026]: as a result, when the conversion event 125 occurs (e.g., the user purchases the particular brand of shoes), Web Site 1 may be at least partially credited for the conversion event although the conversion event 125 may not have occurred directly as a result of the user visiting Web Site 1 (e.g., the user may have visited Web Site 1 two days before the conversion event and/or the user may have navigated to other web sites between visiting Web Site 1 and purchasing the particular brand of shoes). Web Site 1 may be provided attribution credit even though multiple intervening events occurred between the user viewing the advertisement at Web Site 1 and then making the purchase. Under conventional techniques, such events would lead to a result in which the ad impression viewed on Web Site 1 is not credited. Under multi-touch attribution modeling, such as described, the ad impression on Web Site 1 may be counted and thus credited if it appears in the user's browsing history. [0051] Based on this browsing history 400, it may be determined that Web Site 4 440 directly contributed to the conversion event 450 as Web Site 4 440 was visited just before the conversion event 450. Furthermore, Web Site 4 440 has an advertisement that corresponds to the conversion event 450. However, because Web Site 1 410 was also in the user's browsing history and also contained an advertisement that corresponds to the conversion event 450, it may be determined that Web Site 1 410 also contributed to the conversion event 450.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A).
As per claims 8 and 14: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the extent of credit assigned to the event comprises an incremental value associated with the event with respect to the conversion.
Zhang shows: [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0131] Then, for each user, go through each event in the user's event set and add all the subset indexes to a hash and keep track of the counts. For example, for event E1, add the subset indexes of S1 and S2 to a hash; for event E2, add the subset indexes of S1 and S3 to a hash; and for event E3, add the subset indexes of S2 and S3 to a hash. If the hash count of a subset index equals the length of the subset, increase the user count for a subset. [0141] For each user sequence in Table 2, use the inverted index to determine which sub-sequences in Table 3 are subsets of the user sequence, i.e., for which sub-sequences one should increment n.sub.conv and/or n.sub.nonconv. That is, for the first converted user sequence {E.sub.1 E.sub.2 E.sub.3.fwdarw.C}, generate the following list (see Table 5 below) from the inverted index Table 4: {1,3,7,9,10,11; 1,2,8; 2,3,4,6} and then the sub-sequence counts (number of times appearing in the list).
Claims 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhong et al. (2018/0308123), further in view of Curcio et al. (2018/0232264).
As per claim 16:
Regarding the claim limitations below, Zhong in view of Curcio shows:
“A computing system comprising:
determining a hyperparameter associated with a decay rate for use in training a machine learning conditional intensity model, the hyperparameter being determined by fitting a function to a distribution of a time difference of two events of a pair of event types, wherein the two events are from a same event path”
Zhong shows: [0010] In this disclosure, the term “touch point” (also referred to as touchpoint, contact point, or point of contact) refers to any encounter between a consumer and a business. For example, a listener heard an ad about a business on the radio. In this case, the radio represents an offline channel and the ad represents an offline advertising event occurring via the offline channel. Suppose this is the first time the listener encountered the business. This encounter represents an offline touch point. Suppose the listener then went online and visited a website of the business and, while there, made a purchase through the website. The listener's visit to the business's website represents an online touch point. Those skilled in the art will recognize that, whether it is offline or online, a channel can have numerous touch points. [0011] Although it appears that the online touch point resulted in a conversion—the listener made a purchase through the website, in this example, the offline touch point is what caused the listener to visit the business's website in the first place. To be fair and accurate, then, the offline advertising event deserves some credit for the conversion—in other words, this particular conversion should ideally be fractionally attributed to the offline advertising event occurring in the offline channel. However, as the above example illustrates, offline touch points and online touch points may not overlap. Therefore, it can be very difficult, if not impossible, to combine and/or properly associate offline touch points with online touch points.
In light of the description in the spec [0003] “a machine learning conditional intensity model”, Zhong shows [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. Zhong: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0083]-[0089]: credit conversion and fractional credit attribution. Zhong shows “hyperparameter” at least in [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
Even though Zhong shows attribution weight for events and weight distribution to attribute credit correctly in [0083]-[0089], [0119], Zhong does not explicitly show “associated with a decay rate for use”. Curcio shows the above limitation at least in [0028] Generally, individual impressions (or clusters of impressions) in the user's history are analyzed by the attribution engine 120. The attribution model 120 determines which of the impressions (e.g., websites, advertisements) contributed to the conversion event 125. In certain embodiments, each user interaction with an advertisement on a web site is combined with a time function (e.g., a function of a length of time preceding the conversion event) to determine a weighted contribution or attribution value of each of the web sites in the browsing history. For example, a web site visited two days before the conversion event may not be given as much weight as a web site that was visited just one day, hours, or even minutes before the conversion event 125. In another embodiment, the aggregation engine 120 may be configured to determine a chain of web sites that contributed to the conversion event even though the web sites in the chain are not related in subject matter, either to each other, and/or to the subject matter of the conversion event 125. Time-Energy Calculations. [0032] The attribution engine 120 performs calculations on the incoming data. In one embodiment, the attribution engine 120 implements calculations that model each user as a system with a variable amount of energy that, over time, seeks a global baseline state. This “energy” is analogous to, and used as a proxy for, the user's presumed interest level. When viewed as an energy function, acts deemed to be of interest represent energy addition, subject to decay by time (as well as optionally subtraction by other intervening events). Different values in the energy curve indicate the different level of engagement that the user has. The user activities that correlate to the energy values comprise the events that receive attribution for a subsequent conversion event. [0033] More specifically, events in the history preceding the conversion event add or deplete energy from the user's interest level. In particular, one or more embodiments provide that certain events, such as, for example, click events, add energy to the model, while other events, such as, for example, tightly clustered ads, remove energy from the model. Energy injection and extraction for other reasons may be supported via additional types of event meta-data (e.g., ad size, placement, etc.). An attribution decay function is applied to further highlight the change of the model over time. This separate decay is a time effect representing the heuristic that temporally distant events from a conversion event should not be attributed credit for the conversion event or should be attributed less credit for the event. The attribution decay concept more accurately weights past events that contributed to a conversion event than the traditional attribution window. Under an embodiment described, for example, events preceding a conversion event are not modeled for attribution based solely on a timeline or preceding event, but rather analyzed over a preceding duration that takes into account events/activities that peak user interest. [0034] To calculate the energy of a chain, and thus the presumed interest level of a user, at a particular point in time, each event in a user chain is visited in order, from least to most recent. A prophetic example of a chain energy calculation is shown in FIG. 2A. The x-axis of FIG. 2A shows time before a conversion event: the conversion event occurs at time zero, and the farther along the x-axis an event is, the further back in time the event is. The y-axis of FIG. 2A is the modeled chain energy value, which is assumed to reflect the interest level of the user. If an event (e.g., an impression) is determined to be related to the conversion event, the chain energy value at the point in time when the event occurs is applied to the event as a weighting factor. In the example of FIG. 2A, the chain energy has a baseline value of 1.0, and increases (e.g., as a step function) to a value of 2.0 in response to a click event or other user interaction (e.g., in response to a user clicking on, or otherwise interacting with, an impression). After the click event (or other user interaction), the chain energy decreases toward the baseline value. For example, the chain energy decreases based on an exponential decay having a heuristically determined time constant. [0036] In some embodiments, when a conversion event occurs, an attribution decay function is combined with the chain energy function to arrive at the final attribution allotment. The attribution decay may be an exponential function that runs to zero at the end of the client-specified attribution window, as shown by FIG. 2B. The exponential function has a corresponding half-life, the value of which may be determined empirically and may vary from application to application. Examples of half-lives may be in the range of 1-5, or 1-10, or 1-30 or more. Alternatively, the attribution decay may have an alternative form, such as another time-dependent curve.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to account for a decay rate from the first touch point to the next [0028]-[0034], in the disclosure of Reference Zhong, particularly in the ability to attribute weight for events and weight distribution to attribute credit correctly in [0083]-[0089], [0119], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also account for loss of customer interest along the way as taught by Reference Curcio (see at least in [0028]-[0034]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A); and
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“training the machine learning conditional intensity model, using the determined hyperparameter, to identify a causal parameter indicating an extent of excitation of an occurrence of another event.”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n. Zhong shows in the above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight. Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved;
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“generating a causal graph based on the causal parameter, the causal graph including a set of nodes representing event types and a set of directed edges between nodes of the set of nodes representing causal influence between event types, wherein the conditional intensity model and the causal graph are used to identify direct attribution and indirect attribution for an event”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
Zhong also shows “event augmented” in the claim above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight
Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0083]-[0089]: credit conversion and fractional credit attribution.
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does). Zhong shows: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
However, Zhong does not explicitly show the above limitation. Curcio shows the above limitation at least in [0020]: A system 100 includes components for (i) storing user interactions with each of the various web sites and/or an advertisement on the various web sites, (ii) attributing conversion events to the user interactions using a browsing history of each user, and (iii) aggregating data regarding the conversion events to facilitate reporting. [0042] Referring back to FIG. 1, the server 110 may also include an aggregation engine 140 that receives the attributed conversions 145 from the attribution engine 120. The aggregation engine 140 may aggregate the data and transmit the data to a reporting engine 160. The reporting engine 160 may be configured to output the data to an end user for analysis.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A);
Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
“and generating a causal graph based on the causal parameter, the causal graph including a set of nodes representing event types and a set of directed edges between nodes of the set of nodes representing causal influence between event types, wherein the conditional intensity model and the causal graph are used to identify direct attribution and indirect attribution for an event.”
Prior art Zhong shows adjustable parameters [0119], conditional probability [0138], fractional attribution of credit [0083]-[0089], [0091]-[0101]. [0115] To increase confidence levels, one can define events at a less granular level such as the campaign level. There are likely a lot of (both converted and non-converting) users sharing the event of “seeing at least one impression from campaign x”, making the estimates at campaign level more robust. However, if there are only estimates at the campaign level, it does not help to attribute conversion credits across different sites, different frequency or recency values for the same campaign. [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
Zhong also shows “event augmented” in the claim above in [0150] A regression modeling approach can be used to build a predictive model that can predict total (multi-channel) conversions, based on channel volumes. According to embodiments, a what-if analysis to produce a “delta key performance indicator (KPI)” that can be attributed to a given channel. In particular, the what-if analysis sets the volume for a channel to 0 and uses the delta change in predicted conversions as a measure of the conversion contribution from the channel. The deltas may be normalized across all channels to get a channel weight
Zhong shows: [0013]: a new approach is needed to determine accurate conversion credit deserved by each channel and/or campaigns under each channel. The new approach may rely on aggregate-level data to capture causal relationships among different channels and conversions. [0100]-[0107]: shows multiple models – marginal importance model, last click model, [0147]: attribution models, [0150]-[0157]: regression model, [0166]: algorithm; [0175]: aggregate-level regression model. [0015]: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get.
Zhong shows: [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does). Zhong shows: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
Zhong also touches on reporting the information to the user in [0145] After this is done for every conversion, the result is a weight for each impression/click event (i.e., at the most granular level). These final weights can then be rolled up along different dimensions for reporting. Common dimensions of interest include campaign, site, creative, etc.
However, Zhong does not explicitly show the above limitation. Curcio shows the above limitation at least in [0020]: A system 100 includes components for (i) storing user interactions with each of the various web sites and/or an advertisement on the various web sites, (ii) attributing conversion events to the user interactions using a browsing history of each user, and (iii) aggregating data regarding the conversion events to facilitate reporting. [0042] Referring back to FIG. 1, the server 110 may also include an aggregation engine 140 that receives the attributed conversions 145 from the attribution engine 120. The aggregation engine 140 may aggregate the data and transmit the data to a reporting engine 160. The reporting engine 160 may be configured to output the data to an end user for analysis.
Reference Zhong and Reference Curcio are analogous prior art to the claimed invention because the references generally relate to field of attribution models for advertising or marketing campaigns. Further, said references are part of the same classification, i.e., G06Q30/02. Lastly, said references are filed before the effective filing date of the instant application; hence, said references are analogous prior-art references.
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application for AIA to provide the teachings of Reference Curcio, particularly the ability to report the adjusted attribution for the credit to the user at least in [0020] and [0042], in the disclosure of Reference Zhong, particularly in the ability to roll up final weights along different dimensions for reporting in [0145], in order to provide for a system that not only can adjust attribution models so credit is correctly assigned to the correct channels of advertising/marketing, but also report this information back to the user as taught by Reference Curcio (see at least in [0020] and [0042]), where upon the execution of the method and system of Reference Curcio for reporting of the information back to the user, the process of managing attribution models for advertising or marketing campaigns can be made more efficient and effective.
Further, the claimed invention is merely a combination of old elements in a similar attribution models for advertising or marketing campaigns field of endeavor, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Reference Zhong in view of Reference Curcio, the results of the combination were predictable (MPEP 2143 A).
As per claim 17: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the hyperparameter is uniquely determined for the event types of the pair of event types.
Zhang shows “pair of event types” ([0056] 6. Session ID—a number generated on page load by the browser and sent on all requests from that page (Impression, On Load, Post, etc.), used to correlate those events together. [0087] The first desired property is Monotonicity, which means that if two events (e.g., E.sub.1 and E.sub.2) were combined into one composite event E.sub.12 then the fraction credit w.sub.12 for E.sub.12 should most likely be no less than w.sub.1 or w.sub.2. That is, w.sub.12≥w.sub.1 and w.sub.12≥w.sub.2. The intuition is that two events a converted user has with a marketer's campaigns should deserve no less credit than each of those two events individually. [0088] The second property, Correlation with Conversion, holds that the weight for each event should be roughly correlated with the event's ability to drive conversions based on historical data. If E.sub.1 historically has driven conversions better than E.sub.2 and E.sub.3 together, then E.sub.1 deserves more credit than either E.sub.2 and E.sub.3. Zhong shows “hyperparameter” at least in [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function where n is the number of users, and μ and α are adjustable parameters. The parameter μ determines when confidence becomes 0.5 and α controls how fast the confidence grows with n.
As per claim 18: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the trained machine learning conditional intensity model is used to identify direct attribution associated with the event and indirect attribution associated with the event.
Zhong shows: [0015] The new fractional attribution approach is data-driven, without preconceived bias on the importance of different channels. It is also a general approach that works with any number of different types of advertising channels, as long as daily total ad volume is reliably captured. The new approach expands beyond online advertising channels for which cookie-based user level data is available and is able to attribute conversion credit to advertising channels for which no user-level data is available or user-level data is difficult and expensive to get. Embodiments disclosed herein provide an accurate modeling of causal relationship among channels and conversions to thereby determine the most accurate credit to each channel or sub-channel involved. [0143]: FIG. 5 shows attribution weights for a particular user with six impression events before a conversion. The six impressions (imp_1, imp_2, imp_6) are arranged in temporal order. The last-click model assigns all credit to imp_6 whereas an even attribution model assign ⅙ credit to each of the six events. The next two rows show the results of fractional attribution model at campaign level and campaign + frequency level, respectively. In this case, there are four event items for both of those levels but the weights are different as one takes into account frequency in the event definition and the other does not. [0146]: FIG. 6 compares the fractional model with the last-click model and even attribution model, after rolling up the attribution weights to campaign level. Campaign IDs are shown on the x-axis and relative difference between models on the y-axis. For example, for campaign ID 214383 (highlighted in the box), the fractional attribution model assigns to it 12% less credit than last-click model does, but 20% more than even model does).
As per claim 19: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the machine learning conditional intensity model is trained using a set of positive event paths resulting in conversions and a set of negative event paths not resulting in conversions.
Zhang shows [0119] The attribution weight for a given event can be calculated for every node in the hierarchy and combined based on the confidence of each calculation. Confidence can be a function of the amount of data (i.e., the number of users) used to estimate the conditional probabilities. For example, a reasonable confidence function is the sigmoid function. [0138] For each sub-sequence S, count the number of converted users (n.sub.conv) and number of non-converting users (n.sub.nonconv) that have the sub-sequence and compute the conditional probability {w,α,β} are non-negative parameters of the non-linear regression model that are designed to capture interactions between each channel/placement and the KPI and among channel/placements. [0166] The computer may then run a multi-stage least squares regression, as an extension of a two-stage least squares algorithm. Multiple regressions may be run in a stepwise fashion as exemplified below: [0167] a. Assume there are m funnel stages (or levels), going from 1 to m top down. One may first try to determine weights for the bottom (m-th) level channels using the standard two-stage least squares algorithm, treating all channels above the m-th level as instrumental variables. [0168] b. After the causal weights of the m-th level channels are determined, do the same for the (m−1)-th level channels, with the residuals as the target (dependent variable) and the channels in the top (m−2) levels as instrumental variables. [0169] c. Repeat this process until the causal weights are determined for all channels. [0170] d. Optionally, non-negative constraints can be added so that the channel weights cannot be negative.
As per claim 20: Regarding the claim limitations below, Zhong in view of Curcio and Councill shows:
wherein the machine learning conditional intensity model is further trained to identify a baseline parameter that indicates an extent of propensity for the event to occur without any stimulus.
Zhang shows [0091] Embodiments make use of data-driven probabilistic models. That is, all the conditional probability estimates discussed herein are based on historical data. [0092] In particular, each conditional probability P(A|B) can be derived from historical data by dividing the number of users who (at least) had events A and B by number of users who (at least) had event B. That is, [0100] A third model (Model 3) may be the Conditional Importance model. [0101] Consider capturing the importance E.sub.1 by the conditional probability.
Response to Arguments
Applicants’ arguments are moot in view of the new grounds of rejection necessitated by the amendments made to previously presented claims.
Applicant’s Argument #1
Applicants argue on page(s) 10-11 of applicants remarks that prior art does not show the amended claim limitation: “Instead, Applicant's claims are directed to, for example, concrete steps for "determining an adjusted attribution for the event based on the direct attribution for the event augmented with an indirect attribution for the event, the indirect attribution identified, via the abstract idea" into a practical application as explained in Section III.A.2. If the claim as a whole integrates the recited tentative abstract idea into a practical application, the claim is not directed to a judicial exception (Step 2A: NO) and is eligible. Machine learned conditional intensity model," as recited by claim 1. As described in greater detail below, claims 1-3, 5-10, and 12-20 provide a particular solution to problems arising in understanding and capturing causal relationships between different marketing touchpoints and, as a result, sufficiently distributing credit to earlier touchpoints effectuating subsequent touchpoints.” (see applicants remarks for more details).
Response to Argument #1
Applicants' arguments have been fully considered; however, the examiner respectfully disagrees.
Please see Note above. Applicants’ claims are broad and their scope is difficult to reasonably understand by one of ordinary skill in the art. For instance, applicants are arguing that the amended claim limitations have a machine learning model, which is an additional element. The additional elements of a “machine learning model”. This language merely requires execution of an algorithm that can be performed by a generic computer component and provides no detail regarding the operation of that algorithm. As such, the claim requirement amounts to mere instructions to implement the abstract idea on a computer, and, therefore, is not sufficient to make the claim patent eligible. See Alice, 573 U.S. at 226 (“A method comprising: via a machine learned conditional intensity model, one or more computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method comprising: (claim 1). A computing system comprising: in training a machine learning conditional intensity model (claim 16)”); October 2019 Guidance Update at 11–12 (recitation of generic computer limitations for implementing the abstract idea “would not be sufficient to demonstrate integration of a judicial exception into a practical application”). Such a generic recitation of “machine learning model” is insufficient to show a practical application of the recited abstract idea. The additional elements are such that it amounts to no more than: adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f).
Applicant’s Argument #2
Applicants argue on page(s) 10-13 of applicants remarks that prior art does not show the amended claim limitation: “Applicant submits that the claims recite elements that establish a practical application. In Enfish, LLC v. Microsoft Corp., No. 2015-1244 (Fed. Cir. May 12, 2016) (Enfish), the Federal Circuit noted that for computer-related technology a practical application includes a determination as to "whether the claims are directed to an improvement to computer functionality." Enfish at 1335. Furthermore, the Federal Circuit in McRo, Inc. dba Planet Blue v. Bandai Namco Games America, Inc. 120 U.S.P.Q.2d 1091 (Fed. Cir. 2016) (McRo) provided important guidance with respect to whether a claim is directed to an improvement in computer-related technology ... "Advantageously, the causal-based attribution approach models long-term interactions between events as well as the time-decaying relationship between touchpoints and conversions." Id. Therefore, the claims, when read in light of Applicant's specification, the claims provide a specific improvement in computer technology and qualify as eligible subject matter under 35 U.S.C. § 101.” (see applicants remarks for more details).
Response to Argument #2
Applicants' arguments have been fully considered; however, the examiner respectfully disagrees.
Please see Note above. In response, while Examiner acknowledges that the claims include specific computer-implemented functions and analysis of the claimed invention in light of the conclusions of the Courts in the noted DDR Holdings, Core Wireless and Enfish is reasonable, Examiner respectfully disagrees with Applicant’s contention that the claimed features amount to significantly more than the abstract idea or that the claimed invention as presented exhibits technical features/solutions that are reasonably similar to those which the Courts found to be statutory in the noted cases.
In the claims discussed in the 101 rejection, in step 2A prong 1 and step 2B includes various features/limitations that are not directed to the abstract idea. As discussed in the 101 rejection above, in light of the specification, it should be noted that the components discussed above did not meaningfully limit the abstract idea because they merely linked the use of the abstract idea to a particular technological environment (i.e., "implementation via computers"). In light of the specification, it should be noted that the claim limitations discussed above are merely instructions to implement the abstract idea on a computer. See MPEP 2106.05(f).
Response to Arguments
Applicants arguments are moot in view of the new grounds of rejection necessitated by the amendments made to previously presented claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
NPL Reference:
1. P.K. Kannan, Werner Reinartz, Peter C. Verhoef. The path to purchase and attribution modeling: Introduction to special section. International Journal of Research in Marketing. Volume 33, Issue 3, 2016, Pages 449-456, ISSN 0167-8116. https://doi.org/10.1016/j.ijresmar.2016.07.001. (https://www.sciencedirect.com/science/article/pii/S0167811616300817).
Abstract: Firms make significant marketing investments in online, mobile and offline media and channels such as search engines, social media, e-mail, display advertising, print, TV, etc., to draw in customers to their websites, mobile apps, and stores to effect conversions and spur sales. As customers go through a series of touch points across media, channels and devices on their paths to purchase, attributing the appropriate credit for each touch point has emerged as an important problem. By focusing on estimating the incremental value of a touch point and spillover effects across channels, attribution models can provide insights for allocating marketing investments across channels and targeting customers across channels and devices. In this paper, we provide a survey of the state-of-the-art in attribution modeling and analytics. As part of the survey, we also introduce the articles in this special section and position them in our classification framework. Finally, we propose a research agenda to guide future work in the area.
2. Grbovic et al., “E-commerce in You Inbox: Product Recommendations at Scale” supports cited author, see especially [Sect 3.2] parameter 0; Eq (3.4). Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2015), Sydney, Australia. 2015.
[1606.07154] E-commerce in Your Inbox: Product Recommendations at Scale (arxiv.org)
Abstract: In recent years online advertising has become increasingly ubiquitous and effective. Advertisements shown to visitors fund sites and apps that publish digital content, manage social networks, and operate e-mail services. Given such large variety of internet resources, determining an appropriate type of advertising for a given platform has become critical to financial success. Native advertisements, namely ads that are similar in look and feel to content, have had great success in news and social feeds. However, to date there has not been a winning formula for ads in e-mail clients. In this paper we describe a system that leverages user purchase history determined from e-mail receipts to deliver highly personalized product ads to Yahoo Mail users. We propose to use a novel neural language-based algorithm specifically tailored for delivering effective product recommendations, which was evaluated against baselines that included showing popular products and products predicted based on co-occurrence. We conducted rigorous offline testing using a large-scale product purchase data set, covering purchases of more than 29 million users from 172 e-commerce websites. Ads in the form of product recommendations were successfully tested on online traffic, where we observed a steady 9% lift in click-through rates over other ad formats in mail, as well as comparable lift in conversion rates. Following successful tests, the system was launched into production during the holiday season of 2014.
Foreign Reference:
(KR 102639069 B1) KIM MIN WOOK. Artificial Intelligence-based Advertising Method Recommendation System.
Abstract: According to one embodiment, in an artificial intelligence-based advertising method recommendation system, a system including a server that recommends an artificial intelligence-based advertising method is provided.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NANCY PRASAD whose telephone number is (571)270-3265. The examiner can normally be reached M-F: 8:00 AM - 4:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patricia Munson can be reached on (571)270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.N.P/Examiner, Art Unit 3624
/PATRICIA H MUNSON/Supervisory Patent Examiner, Art Unit 3624