Newest papers

The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits

Sharan Sahu, Martin T. Wells · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29982 · cited by 0 works (OpenAlex) · open-access PDF

Korean

그러나, 의사 결정자는 또한 현재 치료 효과에 대한 신뢰할 수있는 추론을 필요로한다 : 개입과 참조 팔 평균 보상 사이의 차이점은 역사적인 지역 평균 대비 효율적입니다. 이러한 목표 충돌 : 보상 최대화는 더 나은 개입을 선호합니다. 그러나, 현재 효과 추론은 시간이 지남에 따라 결과가 변함으로써 잠재적으로 최소한의 개입에 대한 지속적인 탐구를 방지합니다. We characterize this tradeoff for $K$-arm mean rewards at the present time, rather than historical local averages. These objectives conflict: reward maximization favors better interventions, whereas current-effect inference requires continued exploration of potentially suboptimal interventions within width of $w$, while we establish matching current boxes, including logarith

Machine translation (Meta M2M100); the English original follows.

Abstract

Adaptive experiments use bandit algorithms to favor interventions with higher expected rewards. However, decision-makers also require reliable inference on current treatment effects: differences between intervention and reference-arm mean rewards at the present time, rather than historical averages. These objectives conflict: reward maximization favors better interventions, whereas current-effect inference requires continued exploration of potentially suboptimal interventions as outcomes change over time. We characterize this tradeoff for $K$-arm bandits with Hölder-smooth mean rewards of exponent $β\in(0,1]$ and constant $L$. Requiring high-probability simultaneous coverage of all current effects by confidence intervals of width at most $w$, we establish matching minimax bounds, up to logarithmic factors, on expected dynamic regret. Over horizon $T$, our algorithm TRACE (Tracking Rewards and Current Effects) achieves regret of order $\widetilde{O}_β\left(\sqrt{KT}+K^{β/(2β+1)}L^{1/(2β+1)}T^{(β+1)/(2β+1)}+Kw^{-2}(1+L^{1/β}w^{-1/β})\right)$, where the notation hides logarithms and constants depending only on $β$. This regret--precision tradeoff separates reward-learning regret from the cost of refreshing current-effect estimates, which remains unavoidable even when one arm is uniformly optimal. An empirical-Bernstein e-process extension adapts exploration to realized prediction errors while preserving worst-case guarantees. When within-block drift is negligible relative to sampling error, all observations, including reward-learning observations, can be reused for asymptotically normal pointwise inference and stronger local stationarity yields local asymptotic efficiency. Finally, we show that regret constraints can prevent honest adaptation to unknown smoothness and support our theory with extensive numerical experiments.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Regret, Logarithm, Inference, Smoothness, Constant (computer programming), Mathematics

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange