Newest papers

Near Minimax Rate for Quantile Distributional Reinforcement Learning

Zijie Cheng, Zhihua Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29595 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

この論文では、単一の制限値を独立して、Wassersteinインスタンス下の量子分布ポリシーの評価を研究し、あるポリシーの下で回収分布が量子固定リスクポイント$\bmη_m$に近づいている。この論文では、Wasserstein距離下の量子解像度の下で$\bmη_m$を評価するための統計的精度を調査します。生成モデルにアクセスすると、我々は、実験的マルコフ決断プロセスから量子固定リスクポイント$\bmη_m$を構築し、Wasserstein距離下で非アシンプトティックな現地エラー限界を確立します。報酬密度がゼロから均一に制限されているとき、我々は

Machine translation (Meta M2M100); the English original follows.

Abstract

In this paper, we study quantile distributional policy evaluation, where the return distribution under a given policy is approximated by the quantile Bellman fixed point $\bmη_m$. We investigate the statistical accuracy of estimating $\bmη_m$ across quantile resolutions under Wasserstein distances. Assuming access to a generative model, we construct a quantile fixed point estimator $\bmη_m^{(n)}$ from an empirical Markov decision process and establish non-asymptotic error bounds under Wasserstein distances. When reward densities are uniformly bounded away from zero, we obtain a root-$n$ high-probability bound simultaneously over all resolutions $m$ and all Wasserstein orders $1\le p\le\infty$. When reward densities vanish at the support endpoints, we show that continuity and endpoint monotonicity suffice to preserve the same resolution-independent rate for $1\le p\le2$. For $p>2$, we establish a root-$n$ bound with coefficient $m^{1/2-1/p}$, together with a slower, resolution-independent $n^{-1/p}$ bound for every finite $p>2$. We further obtain resolution-independent $W_\infty$ consistency governed by the continuity modulus of reward quantile functions. To investigate the sharpness of these bounds, we characterize the exact local asymptotic minimax risk at each fixed resolution as the expected Wasserstein-coordinate norm of the efficient Gaussian limit and prove that our estimator attains it. We then examine the resolution dependence of this optimal local risk. A regular one-state instance has resolution-independent local risk constants, whereas a single endpoint-vanishing two-state instance exhibits growth of order $m^{1/2-1/p}$, up to logarithmic factors along a resolution subsequence, for every $p>2$.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Mathematics, Quantile, Minimax, Estimator, Wasserstein metric, Bounded function

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange