Newest papers

Next Best Action: Causal Constrained Offline RL for Business Action Targeting

Surya Ramachandiran, Yiming Li, Juan Mancilla-Caceres, Tao Ye, Jun Liu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26817 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

次に、ベストアクション(NBA)システムは、顧客の介入(例えば、割引、プロモーション、ライフサイクルアクション)の中から選択し、長期的な経済的価値を最大化し、複雑な運用制限を満たし、因果的根拠に留まります。既存のアプローチは、これらの要件を共同で満たすための厳格な枠組みを欠いている。我々は、NBAを制限されたマルコフ決定プロセス(CMDP)として明示的に強制することにより、このギャップを解決します。それから、我々は、CMDPの線形空間プログラミング(LP)式に直接推定された顧客動力の非政策因果的な訂正を統合します。結果のポリシーは、顧客の

Machine translation (Meta M2M100); the English original follows.

Abstract

Next Best Action (NBA) systems must select among customer interventions (e.g., discounts, promotions, and lifecycle actions), maximize long-term economic value, satisfy complex operational constraints, and remain causally grounded. Existing approaches lack a rigorous framework for meeting these requirements jointly. We address this gap by formulating NBA as a Constrained Markov Decision Process (CMDP). We then integrate off-policy causal correction of the estimated customer dynamics directly into the Linear Programming (LP) formulation of the CMDP. The resulting policy selects actions based on customer states to maximize long-term value while explicitly enforcing business guardrails. Furthermore, the LP formulation yields shadow prices that quantify the economic trade-offs of operational constraints (e.g., fairness and user welfare), enhancing policy interpretability. We validate our proposed offline Reinforcement Learning (RL) framework on an industry case study involving rideshare coupon targeting, where customer states are defined by Recency, Frequency, and Monetary (RFM) stages. We evaluate performance using model-based off-policy evaluation, with causal dynamics estimated from a temporal holdout. Under the observed coupon budget and RFM state representation, our optimized policy improves customer lifetime value (LTV) by 19% over the estimated behavior policy and 8% over a myopic baseline. Although the demonstrated action space is binary, the proposed framework is action-agnostic, yielding a general-purpose decision module for targeting systems that manage multiple interventions.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Markov decision process, Computer science, Reinforcement learning, Markov process, State space, Operations research

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange