Net Returns Maximization¶
This section covers budget optimization by maximizing net returns with the insights from a fitted MMM and a user supplied LTV analysis (in channel level). Behind the scene, we use the SLSQP method from scipy.optim. It is recommended to learn basic from the scipy manual.
In net returns maximization, the objective function is define as
\begin{align*} R & = \sum^K_k (\text{LTV}_k - \text{Cost per Acqusition}_k) \times \text{Attribution}_k \\ & = \sum^K_k (\text{LTV}_k - \frac{\text{Spend}_k}{\text{Attribution}_k}) \times \text{Attribution}_k \\ & = \sum^K_k (\text{LTV}_k \times \text{Attribution}_k - \text{Spend}_k) \end{align*}
import numpy as np
import pandas as pd
import math
import matplotlib.pyplot as plt
import pickle
from copy import deepcopy
# for version print
import orbit
import scipy
from karpiu.planning.optim import ChannelNetProfitMaximizer, TimeNetProfitMaximizer
from karpiu.planning.optim.time_base_optimizer import (
time_based_net_profit_response_curve,
)
from karpiu.planning.optim.channel_optimizer import ch_based_net_profit_response_curve
from karpiu.planning.common import calculate_marginal_cost, generate_cost_report
from karpiu.explainability import AttributorGamma as Attributor
pd.set_option("display.float_format", lambda x: "%.5f" % x)
%load_ext autoreload
%autoreload 2
print(orbit.__version__)
print(pd.__version__)
print(np.__version__)
print(scipy.__version__)
1.1.4.2 1.4.2 1.24.2 1.10.0
We reload the fitted model from previous Quickstart section here.
with open("./resource/seasonal/model.pkl", "rb") as f:
mmm = pickle.load(f)
Demo on Bivariate Inputs¶
budget_start = pd.to_datetime("2021-01-01")
budget_end = pd.to_datetime("2021-01-02")
optim_channels = mmm.get_spend_cols()[1:3]
optim_channels
['radio', 'search']
Given the customers life-time value (LTV), one can run the revenue maximization.
# provide some arbitrary ltv arrays
# they need to be in full channels length
ltv_arr = [48.5, 52.5, 38.6, 35.8, 60.8]
temp_mmm = deepcopy(mmm)
attributor = Attributor(model=temp_mmm, start=budget_start, end=budget_end)
t_npm = TimeNetProfitMaximizer(
ltv_arr=ltv_arr,
model=temp_mmm,
attributor=attributor,
budget_start=budget_start,
budget_end=budget_end,
optim_channels=optim_channels,
spend_scaler=1.0,
response_scaler=1.0,
)
2023-12-11 19:02:47 - karpiu-planning - INFO - Full calculation start=2021-01-01 and end=2021-01-02 2023-12-11 19:02:47 - karpiu-planning - INFO - Attribution start=2021-01-01 and end=2021-01-02 2023-12-11 19:02:47 - karpiu-planning - INFO - Optimizing channels : ['radio', 'search']
We can leverage the module time_based_net_profit_response_curve (only works for bivariate inputs) to plot the surface of the response and then add the track footprint from each optimization step.
%%time
x, y, z = time_based_net_profit_response_curve(t_npm=t_npm, model=temp_mmm, n_iters=10)
fig, ax = plt.subplots(1, 1, figsize=(12, 8))
cs = ax.contourf(x, y, z, 20, cmap="RdGy")
fig.colorbar(cs, ax=ax)
ax.set_xlabel("t1")
ax.set_ylabel("t2")
ax.plot(
[0, t_npm.total_budget],
[t_npm.total_budget, 0],
alpha=0.5,
label="total budget constraint",
linestyle="--",
)
# run optim steps
t_optim_df = t_npm.optimize(maxiter=500)
# use callback metrics to extract solutions on each step
callback_metrics = t_npm.get_callback_metrics()
xs = np.array(callback_metrics["xs"])
ax.arrow(
x=xs[0, 0],
y=xs[0, 1],
dx=xs[-1, 0] - xs[0, 0],
dy=xs[-1, 1] - xs[0, 1],
width=80,
color="green",
)
fig.legend(loc="lower center")
fig.tight_layout()
Optimization terminated successfully (Exit mode 0)
Current function value: -11646.745792149977
Iterations: 24
Function evaluations: 73
Gradient evaluations: 24
CPU times: user 3.96 s, sys: 22.1 ms, total: 3.98 s
Wall time: 3.98 s
The ChannelNetProfitMaximizer share similar interface to perform channel based optimization.
temp_mmm = deepcopy(mmm)
ch_npm = ChannelNetProfitMaximizer(
ltv_arr=ltv_arr,
model=temp_mmm,
attributor=attributor,
optim_channels=optim_channels,
budget_start=budget_start,
budget_end=budget_end,
spend_scaler=1.0,
response_scaler=1.0,
)
2023-12-11 19:02:51 - karpiu-planning - INFO - Optimizing channels : ['radio', 'search']
%%time
# we need higher iteration to support higher resolution on channels response surface
x, y, z = ch_based_net_profit_response_curve(ch_npm=ch_npm, model=temp_mmm, n_iters=20)
fig, ax = plt.subplots(1, 1, figsize=(12, 8))
cs = ax.contourf(x, y, z, 20, cmap="RdGy")
fig.colorbar(cs, ax=ax)
ax.set_xlabel("t1")
ax.set_ylabel("t2")
ax.plot(
[0, ch_npm.total_budget],
[ch_npm.total_budget, 0],
alpha=0.5,
label="total budget constraint",
linestyle="--",
)
# run optim steps
cn_optim_df = ch_npm.optimize(maxiter=500)
# use callback metrics to extract solutions on each step
callback_metrics = ch_npm.get_callback_metrics()
xs = np.array(callback_metrics["xs"])
ax.arrow(
x=xs[0, 0],
y=xs[0, 1],
dx=xs[-1, 0] - xs[0, 0],
dy=xs[-1, 1] - xs[0, 1],
width=80,
color="green",
)
fig.legend(loc="lower center")
fig.tight_layout()
Optimization terminated successfully (Exit mode 0)
Current function value: -17929.022112135015
Iterations: 11
Function evaluations: 33
Gradient evaluations: 11
CPU times: user 15.4 s, sys: 63.8 ms, total: 15.5 s
Wall time: 15.5 s
Two-Stage Optimization¶
We can also chain the two stages together. Let's see what happen. In practice, one can iteratively call these two methods.
budget_start = pd.to_datetime("2021-01-01")
budget_end = pd.to_datetime("2021-03-31")
optim_channels = mmm.get_spend_cols()
Given the customers life-time value (LTV), one can run the revenue maximization.
Optimizing Channel Budget¶
First, we optimize the budget mix.
%%time
attributor = Attributor(model=temp_mmm, start=budget_start, end=budget_end)
ch_npm = ChannelNetProfitMaximizer(
ltv_arr=ltv_arr,
model=temp_mmm,
attributor=attributor,
optim_channels=optim_channels,
budget_start=budget_start,
budget_end=budget_end,
spend_scaler=1.0,
response_scaler=1.0,
)
2023-12-11 19:03:06 - karpiu-planning - INFO - Full calculation start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:06 - karpiu-planning - INFO - Attribution start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:06 - karpiu-planning - INFO - Optimizing channels : ['promo', 'radio', 'search', 'social', 'tv']
CPU times: user 192 ms, sys: 2.64 ms, total: 194 ms Wall time: 64.4 ms
temp_optim_spend_df = ch_npm.optimize(maxiter=500, eps=1e-3)
Optimization terminated successfully (Exit mode 0)
Current function value: -353153.70395099116
Iterations: 74
Function evaluations: 444
Gradient evaluations: 74
# expermential plot the revs and costs
fig, ax = plt.subplots(1, 1, figsize=(16, 8))
callback_metrics = ch_npm.get_callback_metrics()
optim_revenues = np.array(callback_metrics["optim_revenues"])
optim_costs = np.array(callback_metrics["optim_costs"])
ax.plot(optim_revenues, label="revs")
ax.plot(optim_costs, label="costs")
ax.plot(optim_revenues - optim_costs, label="net-profit")
ax.set_xlabel("iterations")
ax.set_ylabel("metrics")
fig.legend()
fig.tight_layout();
We can retreive the initial and final state of the optimization.
channel_spend_arr = ch_npm.get_current_state()
init_spend_arr = ch_npm.get_init_state()
init_spend_matrix = ch_npm.get_init_spend_matrix()
# total spend suggest
print("Suggested total spend: {:.0f}".format(np.sum(channel_spend_arr)))
# total budget
print("Total Budget: {:.0f}".format(np.sum(init_spend_arr)))
Suggested total spend: 1327092 Total Budget: 1327092
Optimizing Budget Allocation across Time¶
Second, given the condition of budget mix, we optimize the allocation across time.
%%time
temp_mmm = deepcopy(mmm)
temp_mmm.raw_df = temp_optim_spend_df
t_npm = TimeNetProfitMaximizer(
ltv_arr=ltv_arr,
model=temp_mmm,
attributor=attributor,
budget_start=budget_start,
budget_end=budget_end,
optim_channels=optim_channels,
spend_scaler=1.0,
response_scaler=1.0,
)
2023-12-11 19:03:07 - karpiu-planning - INFO - Full calculation start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:07 - karpiu-planning - INFO - Attribution start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:07 - karpiu-planning - INFO - Optimizing channels : ['promo', 'radio', 'search', 'social', 'tv']
CPU times: user 197 ms, sys: 4.24 ms, total: 201 ms Wall time: 69.6 ms
optim_spend_df = t_npm.optimize(maxiter=500)
Optimization terminated successfully (Exit mode 0)
Current function value: -585281.2963156031
Iterations: 371
Function evaluations: 33671
Gradient evaluations: 370
optim_spend_matrix = t_npm.get_current_spend_matrix()
time_based_spend_arr = t_npm.get_current_state()
# total spend suggest
print("Suggested total spend: {:.0f}".format(np.sum(optim_spend_matrix)))
# total budget
print("Total Budget: {:.0f}".format(np.sum(init_spend_matrix)))
Suggested total spend: 1327092 Total Budget: 1327092
Validation¶
Average and Marginal Cost Change¶
In general, one should expect when LTV is greater than the marginal cost under the pre-optimized spend, there should be an increase of spend in the optimal budget.
df = mmm.get_raw_df()
cost_report = generate_cost_report(
model=mmm,
start=budget_start,
end=budget_end,
pre_spend_df=df,
post_spend_df=optim_spend_df,
)
cost_report["ltv"] = ltv_arr
cost_report
2023-12-11 19:03:11 - karpiu-planning - INFO - Full calculation start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:11 - karpiu-planning - INFO - Attribution start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:11 - karpiu-planning - INFO - Full calculation start=2021-01-01 and end=2021-03-31 2023-12-11 19:03:11 - karpiu-planning - INFO - Attribution start=2021-01-01 and end=2021-03-31
| pre-opt-avg-cost | post-opt-avg-cost | pre-opt-marginal-cost | post-opt-marginal-cost | pre-opt-spend | post-opt-spend | pre-opt-attr | post-opt-attr | ltv | |
|---|---|---|---|---|---|---|---|---|---|
| promo | 41.56216 | 28.70818 | 41.28361 | 45.01549 | 145.95900 | 153.31830 | 3511.82429 | 5340.57816 | 48.50000 |
| radio | 65.77044 | 32.33169 | 49.65129 | 50.74166 | 294.95500 | 179.14091 | 4484.61328 | 5540.72241 | 52.50000 |
| search | 27.12460 | 25.31975 | 23.30565 | 35.27171 | 240.82900 | 549.17303 | 8878.61992 | 21689.51500 | 38.60000 |
| social | 29.48616 | 22.48012 | 23.57794 | 33.45444 | 171.27100 | 273.80843 | 5808.52155 | 12180.02265 | 35.80000 |
| tv | 89.58398 | 38.09382 | 76.53350 | 59.44528 | 474.07800 | 171.65133 | 5291.99524 | 4506.01507 | 60.80000 |
pre_mc = cost_report["pre-opt-marginal-cost"].values
overspend = pre_mc > (cost_report["ltv"].values * 1.2)
underspend = pre_mc < (cost_report["ltv"].values * 0.8)
spend_delta = cost_report["post-opt-spend"].values - cost_report["pre-opt-spend"].values
assert np.all(spend_delta[overspend] < 0)
assert np.all(spend_delta[underspend] > 0)
After all, a general condition of marginal cost lower than LTV should be met when spend > 0.
post_mc = cost_report["post-opt-marginal-cost"].values
assert np.all(post_mc < cost_report["ltv"].values * 1.1)
Spend Allocation Plot¶
Channels mix comparison
init_total_spend = np.sum(init_spend_matrix, 0)
optim_total_spend = np.sum(optim_spend_matrix, 0)
plot_data = np.vstack([init_total_spend, optim_total_spend])
plot_data.shape
fig, ax = plt.subplots(1, 1, figsize=(20, 10))
bottom = 0.0
for idx, label in enumerate(optim_channels):
ax.bar(x=["Pre", "Post"], height=plot_data[:, idx], bottom=bottom, label=label)
bottom += plot_data[:, idx]
ax.set_title("Spend Allocation Pre vs. Post Optimization")
ax.legend(
loc="lower center",
bbox_to_anchor=(0.5, -0.10),
ncol=math.ceil(len(optim_channels) / 2),
);
One way to validate time-base budget is to compare with the organic. Due to the multiplicative properties, they should be propertional to each other.
The MMMShell object serves a snapshot purpose of a MMM object.
from karpiu.model_shell import MMMShell
msh = MMMShell(temp_mmm)
init_df = temp_mmm.get_raw_df()
organic_attr_arr = msh.attr_organic[
(init_df[temp_mmm.date_col] >= budget_start)
& (init_df[temp_mmm.date_col] <= budget_end)
]
t_based_allocation_comp = t_npm.get_current_state()
fig, ax = plt.subplots(1, 1, figsize=(16, 8))
ax.plot(
pd.date_range(budget_start, budget_end, freq="1D"),
t_based_allocation_comp / np.sum(t_based_allocation_comp),
label="MMM-Time-Based-Budget%",
alpha=0.5,
)
ax.plot(
pd.date_range(budget_start, budget_end, freq="1D"),
organic_attr_arr / np.sum(organic_attr_arr),
label="MMM-Forecasted-Organic",
alpha=0.5,
)
fig.legend()
fig.tight_layout()
Customize Budget Constraints and Bounds¶
Channels mix¶
The channel "total" is a special reserved to specify total budget constraint. When lower bound and upper bound are equal, it enforce the budget total fixed.
bounds_and_constraints_df = pd.DataFrame(
{
"channels": optim_channels + ["total"],
"upper": np.array(
[291918.0, 589910.0, 481658.0, 342542.0, 948156.0, 1327092.0]
),
"lower": np.array([14595.9, 29495.5, 24082.9, 17127.1, 47407.8, 1327092.0]),
}
)
bounds_and_constraints_df
| channels | upper | lower | |
|---|---|---|---|
| 0 | promo | 291918.00000 | 14595.90000 |
| 1 | radio | 589910.00000 | 29495.50000 |
| 2 | search | 481658.00000 | 24082.90000 |
| 3 | social | 342542.00000 | 17127.10000 |
| 4 | tv | 948156.00000 | 47407.80000 |
| 5 | total | 1327092.00000 | 1327092.00000 |
ch_npm.set_bounds_and_constraints(df=bounds_and_constraints_df)
2023-12-11 19:03:12 - karpiu-planning - INFO - Set bounds. 2023-12-11 19:03:12 - karpiu-planning - INFO - Set total budget constraints.
Time-based allocaton¶
Similar to channels mix, "total" is a reserved keyword.
bounds_and_constraints_df = pd.DataFrame(
{
"date": t_npm.df.loc[t_npm.budget_mask, "date"].astype(str).to_list()
+ ["total"],
"upper": np.concatenate(
[np.ones(t_npm.n_budget_steps) * 30000, np.ones(1) * 1327092]
),
"lower": np.concatenate([np.zeros(t_npm.n_budget_steps), np.ones(1) * 1327092]),
}
)
bounds_and_constraints_df
| date | upper | lower | |
|---|---|---|---|
| 0 | 2021-01-01 | 30000.00000 | 0.00000 |
| 1 | 2021-01-02 | 30000.00000 | 0.00000 |
| 2 | 2021-01-03 | 30000.00000 | 0.00000 |
| 3 | 2021-01-04 | 30000.00000 | 0.00000 |
| 4 | 2021-01-05 | 30000.00000 | 0.00000 |
| ... | ... | ... | ... |
| 86 | 2021-03-28 | 30000.00000 | 0.00000 |
| 87 | 2021-03-29 | 30000.00000 | 0.00000 |
| 88 | 2021-03-30 | 30000.00000 | 0.00000 |
| 89 | 2021-03-31 | 30000.00000 | 0.00000 |
| 90 | total | 1327092.00000 | 1327092.00000 |
91 rows × 3 columns
t_npm.set_bounds_and_constraints(bounds_and_constraints_df)
2023-12-11 19:03:12 - karpiu-planning - INFO - Set bounds. 2023-12-11 19:03:12 - karpiu-planning - INFO - Set total budget constraints.