How to use the bandit dashboard

What is a multi-arm bandit in this project?

A multi-armed bandit repeatedly chooses one action from a set and learns only from the action it chose. Here, an arm is an email angle, the context is the audience segment, and the reward is a booked meeting. Unlike a fixed A/B test, the traffic allocation changes after every observation.

Each tick samples one plausible CTR, \(\theta_i\), from every eligible arm's current posterior and selects \(\arg\max_i \theta_i\). This randomized policy naturally sends more traffic to strong arms while still testing uncertain ones. The Stanford Thompson Sampling tutorial provides the algorithmic background.

The Gate is a hard eligibility filter applied before optimization. Unsafe arms can remain visible for auditability but cannot enter the candidate set. Treat this like an allowlist boundary: policy decides what may compete; the bandit optimizes only inside that set.

Intervention controls

These controls mutate experiment state. Inject creates a new arm with a \(\operatorname{Beta}(1,1)\) prior, so it begins maximally uncertain and will receive exploratory traffic. Pause stops new observations without changing stored state. Fast-forward sends the current state to the backend, executes exactly 1,000 normal decision/update cycles, and applies the returned state atomically; it is a convergence test, not synthetic aggregation.

Pace changes only wall-clock timing, not the random process or posterior math. Reset is destructive to in-memory evidence: counters, posterior parameters, regret, and logs return to initialization. Gate masking changes the eligible action set before sampling; turning it off intentionally allows planted unsafe arms to compete.

Simulation traffic pulse

Sends is the global denominator across all segments and arms. Meetings booked is the global positive-reward count; \(47/248 = 19.0\%\) is a useful aggregate health check, but it should not be used to rank arms because traffic is adaptively allocated and segments differ.

List burn counts unsubscribes. The alert rate is \(r_{\text{unsub}}=N_{\text{unsub}}/N_{\text{sends}}\), with a warning above \(0.50\%\). At the posterior level, an unsubscribe adds three \(\beta\) pseudo-counts, encoding that reputational harm costs more than an ordinary miss.

Gate veto count is an observability metric, not a denominator in CTR. It shows how often the demo exercised its safety boundary. A high count can mean the control is working under adversarial pressure; it does not by itself indicate degraded campaign performance.

Live belief curves and statistical confidence

Each curve is a Beta distribution: the bandit's current belief about one copy arm's true meeting-booking rate. The x-axis is possible CTR. The curve's shape tells you confidence.

• Wide curve: not enough data yet. The arm may still be good or bad.
• Tall narrow curve: enough evidence has accumulated. The model is more certain.
• Curve shifted right: the arm is believed to have higher CTR.
• Overlapping curves: no clear winner yet; exploration should continue.
• Hollow circle: one Thompson sample drawn from that arm's curve for the current decision. The sample changes on every simulated send.
• Circle with an outer ring: the winning sample. The bandit serves the arm whose sampled CTR is highest; the label above it explains whether that choice exploits the current leader or explores a challenger.
• Truth and dashed line: the arm's fixed underlying CTR in this demo simulation. It generates the simulated outcomes and lets you compare the model's belief with reality. In a production experiment, this value is unknown and must be learned from observed results.

The y-axis is probability density, not probability. A taller peak does not mean a higher CTR; horizontal position indicates CTR, while narrowness indicates concentration. The NIST Beta distribution reference covers the density and shape parameters.

Exploit winner vs explore underdog

The meter classifies historical decisions, not a configured exploration rate. A pull is exploit when the sampled winner is also the arm with the highest posterior mean at that tick. It is explore when uncertainty gives another arm the highest random Thompson sample.

Thompson sampling has no fixed epsilon. Exploration emerges from posterior overlap: wide distributions occasionally produce high draws. As evidence accumulates, weak-arm distributions narrow and their chance of winning a draw falls.

Persistent exploration is not automatically a bug. It can indicate statistically similar arms, sparse traffic, a non-stationary reward process, or noisy outcomes. Investigate posterior overlap and segment quality before forcing exploitation.

Candidate copy arms

Each card shows the operational and statistical state of a copy angle.

• Claim tags: the evidence-backed claims the message relies on.
• Belief \(\operatorname{Beta}(\alpha,\beta)\): \(\alpha\) grows with meetings; \(\beta\) grows with misses and more sharply with unsubscribes.
• \(P(\text{Best})\): Monte Carlo estimate of how often this arm beats the others.
• 95% credible interval: the plausible range for this arm's CTR based on current evidence.
• Archive copy: removes the arm from future pulls without deleting its history.

Worked example

Start with the excerpt as an engineer would see it in the dashboard, then separate observed state from inferred state.

1. Separate copy metadata from experiment data

22%

A verified claim embedded in the email body. It is product evidence, not observed conversion data and not an input to the posterior update.

20 minutes

The proposed meeting duration in the CTA. It also has no statistical role.

3 sends

The number of times this arm was selected for the current segment. This is the arm-local sample count, also shown as \(N=3\).

1 click

One send produced the positive reward event. The UI says click, while the experiment interprets it as a booked-meeting success.

2. Reconstruct the posterior

The model starts each arm with a uniform \(\operatorname{Beta}(1,1)\) prior over its unknown conversion probability \(\theta_A\). Beta is conjugate to the Bernoulli likelihood, so ordinary binary outcomes update its two shape parameters as pseudo-counts:

The raw observed rate is \(1/3=33.3\%\). The displayed estimate is the posterior mean:

The difference is prior smoothing. With only three observations, the two prior pseudo-counts still have substantial influence. As traffic grows, the likelihood dominates and the posterior mean approaches the empirical rate.

Implementation nuance: an unsubscribe applies \(\beta\leftarrow\beta+3\) instead of \(\beta\leftarrow\beta+1\). That is a deliberate utility penalty equivalent to three negative pseudo-observations. Once weighted outcomes are used, interpret \(\operatorname{Beta}(\alpha,\beta)\) as a decision model, not a literal posterior from only independent Bernoulli trials.

3. Read absolute uncertainty

\([6.7\%,80.3\%]\) approximates the central 95% posterior credible interval. The dashboard takes 1,000 samples from \(\operatorname{Beta}(2,3)\), sorts them, and uses draws near \(Q_{0.025}\) and \(Q_{0.975}\). Conditional on the model, roughly 95% of posterior probability lies between those bounds:

This is not a frequentist confidence interval and does not mean 95% of future clicks land in that range. It describes uncertainty about the single latent CTR parameter. The interval is extremely wide because three sends contain little information.

4. Read relative ranking

\(P(\text{Best}_A)=84.5\%\) answers a different question from estimated CTR. For each of 1,000 Monte Carlo rounds, the dashboard draws one \(\theta_i\) from every eligible arm and records which draw is largest. Angle A won about \(845/1000\) rounds.

This probability depends on the competing arms and the current Gate/archive mask. It can change even if Angle A receives no new data, because another arm changed or left the pool. At \(\hat p=0.845\) and \(R=1000\) draws, the Monte Carlo standard error is approximately

That is roughly 1.1 percentage points, so small refresh-to-refresh changes are simulation noise.

5. Read policy state

Learned leader

The eligible arm with the highest current \(P(\text{Best})\), not a declaration that learning is complete. Wide intervals can coexist with a leader when competitors are even less promising.

\(M_A=0\)

Archiving sets this arm's action mask to zero. Thompson sampling stops selecting it, but \(N\), \(\alpha\), \(\beta\), and historical outcomes remain intact for auditability or restoration.

Real-time decision stream

The feed is an append-only audit view displayed newest-first. Each row records the segment, selected arm, decision classification, sampled \(\theta_i\), timestamp, and observed outcome. The sampled percentage is the random Thompson draw that won this tick; it is not the arm's estimated CTR.

Follow one row through the state transition: selection increments sends; a meeting applies \(\alpha\leftarrow\alpha+1\); an ordinary miss applies \(\beta\leftarrow\beta+1\); an unsubscribe applies \(\beta\leftarrow\beta+3\). The dashboard then recomputes confidence and redraws the posterior for the affected segment.

Use the feed to explain discontinuities. If an arm's estimated CTR drops sharply, inspect recent outcomes rather than assuming a rendering defect. Repeated unsubscribes should produce a faster leftward update than repeated ordinary misses because of the weighted penalty.

Specialist diagnostics

Interpret the example values

Estimated regret accumulates the simulated opportunity cost of each choice:

A value of \(R_T=1.7\) means 1.7 expected successes were forgone across the run. This is available only because the demo knows each arm's hidden truth, \(\theta_i\); production cannot compute true regret directly.

Policy entropy is computed over each active arm's \(P(\text{Best})\):

It measures how diffuse the current ranking is. For two arms, entropy approaches \(0\) when one dominates and peaks at \(\log 2\approx0.693\) when both are equally likely to be best. Entropy depends on pool size, \(K\), so compare it only with the same eligible-arm count.

Average KL divergence measures how far each posterior has moved from its \(\operatorname{Beta}(1,1)\) prior, then averages across active arms:

Higher means more belief change, not necessarily better performance. KL is asymmetric and measured in nats here. SciPy's entropy and relative-entropy reference gives the definitions.

Active arm pool is the count after archive and Gate masks. It is the denominator for competition and changes the interpretation of entropy and \(P(\text{Best})\).

Head-to-head win probability matrix

Each Monte Carlo round draws one plausible CTR from every active posterior. Cell \((A,B)=78.4\%\) estimates \(P(\theta_A>\theta_B)\): A's draw exceeded B's in roughly 784 of 1,000 rounds. With continuous Beta distributions, ties have probability zero, so

The diagonal is undefined because comparing an arm with itself has no decision value. This matrix answers pairwise superiority, while \(P(\text{Best})\) asks whether an arm beats all competitors simultaneously. With three or more arms, \(P(\theta_A>\theta_B)\) can be high while A's \(P(\text{Best})\) remains modest because arm C often wins.

Treat 70% highlighting as a dashboard heuristic, not a universal statistical threshold. Use the interval width, business cost, and sample volume when deciding whether the evidence is actionable.

Posterior parameter table

Interpret the example values

Turn the table into an action

1. Check volume first: confirm sends and outcomes are large enough to support interpretation. Sparse rows produce unstable means and boundary modes.
2. Use mean for direction: identify which arm currently has the higher expected CTR.
3. Use variance and the credible interval for confidence: determine whether the apparent gap is stable or still dominated by uncertainty.
4. Use \(P(\text{Best})\) and the win matrix for comparison: verify that the preferred arm usually beats the alternatives across posterior draws.
5. Check business constraints: review unsubscribes, approved claims, audience fit, and the cost of sending weak copy.
6. Choose the least destructive action: continue learning when evidence is weak, favor the leader when evidence is directional, and archive only when evidence and business context agree.

Mean is the expected CTR under the posterior and the estimate used to identify the current mean leader. Mode is the peak location when both parameters exceed one:

Mean and mode differ for skewed distributions.

Variance measures posterior spread:

Lower variance means the posterior is more concentrated, but compare variance alongside the mean because probabilities near zero or one naturally have different scale.

KL divergence quantifies information gained relative to the uniform prior. It does not encode direction: a confidently poor arm can have higher KL than a promising but uncertain arm. The table is the numeric representation of the same distributions drawn in the belief chart.

Posterior drift log

Each entry is a compact state snapshot after a pull: timestamp, segment, arm, KL divergence, updated posterior mean, and \(P(\text{Best})\). Read adjacent entries as transitions, not independent events.

One success increments \(\alpha\), usually moving the mean right. One miss increments \(\beta\), moving it left. Either outcome generally reduces \(\operatorname{Var}(\theta)\) as effective evidence increases. An unsubscribe increments \(\beta\) by three, so both the mean and \(D_{\mathrm{KL}}\) can move more sharply.

\(P(\text{Best})\) may move even when the displayed arm was not selected because all arms are re-compared after relevant segment updates. The log retains only the newest 50 entries, so it is an operational debugging buffer rather than durable experiment storage.

Scenario examples and actions

Scenarios combine several signals. Avoid making operational decisions from one KPI: first confirm eligibility, then inspect outcome quality, posterior uncertainty, and convergence.

Scenario 1: CFO ROI proof is clearly winning

Evidence: high \(P(\text{Best})\), a narrow credible interval, low entropy, and mostly exploit pulls. Action: keep it active and consider archiving weak challengers only after checking their send counts. Promotion still requires copy and claim review; statistical rank does not validate message truth.

Scenario 2: Transparent pricing has few sends and a wide interval

Evidence: lower posterior mean, low N, and substantial interval overlap. Action: do not archive based on point estimate alone. Continue normal exploration or fast-forward the demo, then reassess whether uncertainty actually contracts.

Scenario 3: Planted lie has the highest latent CTR

Evidence: the unsafe arm's simulated truth is high, but it is marked Gate-vetoed and absent from sampling. Action: keep masking enabled. This demonstrates separation of concerns: evidence policy controls eligibility, while the optimizer controls allocation.

Scenario 4: List burn spikes

Evidence: global unsubscribe rate exceeds 0.50%, with recent unsubscribe outcomes concentrated in the feed. Action: pause, identify the contributing segment and arm, inspect absolute counts as well as rate, then archive the arm if the harm is concentrated. A one-event spike at tiny N needs different treatment from sustained burn.

Scenario 5: Entropy remains high after many sends

Evidence: \(P(\text{Best})\) remains diffuse and posterior curves overlap despite substantial traffic. Action: check reward latency, non-stationarity, broad segments, and near-identical arms. More traffic does not fix a misspecified reward or mixed population; consider splitting the segment or redesigning the challenger.