Newest papers

Next Best Action: Causal Constrained Offline RL for Business Action Targeting

Surya Ramachandiran, Yiming Li, Juan Mancilla-Caceres, Tao Ye, Jun Liu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26817 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

下一步,客户行为最佳(NBA)系统的目标是选择客户干预(例如,折扣,促销和生命周期行动),最大化长期经济价值,满足复杂的运营限制,并保持因果性。现有的方法缺乏共同满足这些要求的严格框架。我们通过制定NBA作为限制马尔科夫决策流程(CMDP)来解决这个差距,然后我们将估计客户动态的非政策因果性纠正直接整合到CMDP的线性空间编程(LP)公式中。由此产生的政策选择了基于客户状态的干预措施,以最大限度地提高长期价值,同时明确地执行业务防护措施。此外,LP公式还将计量运营限制(例如,公平和用户福利)的经济因果性纠正,从而提高政策的解释性。我们通过对行业的

Machine translation (Meta M2M100); the English original follows.

Abstract

Next Best Action (NBA) systems must select among customer interventions (e.g., discounts, promotions, and lifecycle actions), maximize long-term economic value, satisfy complex operational constraints, and remain causally grounded. Existing approaches lack a rigorous framework for meeting these requirements jointly. We address this gap by formulating NBA as a Constrained Markov Decision Process (CMDP). We then integrate off-policy causal correction of the estimated customer dynamics directly into the Linear Programming (LP) formulation of the CMDP. The resulting policy selects actions based on customer states to maximize long-term value while explicitly enforcing business guardrails. Furthermore, the LP formulation yields shadow prices that quantify the economic trade-offs of operational constraints (e.g., fairness and user welfare), enhancing policy interpretability. We validate our proposed offline Reinforcement Learning (RL) framework on an industry case study involving rideshare coupon targeting, where customer states are defined by Recency, Frequency, and Monetary (RFM) stages. We evaluate performance using model-based off-policy evaluation, with causal dynamics estimated from a temporal holdout. Under the observed coupon budget and RFM state representation, our optimized policy improves customer lifetime value (LTV) by 19% over the estimated behavior policy and 8% over a myopic baseline. Although the demonstrated action space is binary, the proposed framework is action-agnostic, yielding a general-purpose decision module for targeting systems that manage multiple interventions.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Markov decision process, Computer science, Reinforcement learning, Markov process, State space, Operations research

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange