Newest papers

Next Best Action: Causal Constrained Offline RL for Business Action Targeting

Surya Ramachandiran, Yiming Li, Juan Mancilla-Caceres, Tao Ye, Jun Liu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26817 · cited by 0 works (OpenAlex) · open-access PDF

Korean

다음은 베스트 액션 (NBA) 시스템은 고객 개입 (예를 들어 프레임, 프로모션 및 라이프 사이클 액션) 중에서 선택해야하며, 장기 경제적 가치를 극대화하고 복잡한 운영 제약을 충족시키고 원인적 근거를 유지합니다. 기존 접근 방식은 이러한 요구 사항을 공동으로 충족시키기위한 엄격한 프레임을 부족합니다. NBA를 제한 마르코프 결정 프로세스 (CMDP)로 정의함으로써이 격차를 해결합니다. 우리는 CMDP의 선형 공간 프로그래밍 (LP) 공식화에 직접 예상되는 고객 동력의 원인적 교정을 통합합니다. 결과 정책은 고객 조건에 근거하여 조치를 선택하여 장기 가치를 극대화합니다. 또한 LP 공식화는 운영 제한의 경제적

Machine translation (Meta M2M100); the English original follows.

Abstract

Next Best Action (NBA) systems must select among customer interventions (e.g., discounts, promotions, and lifecycle actions), maximize long-term economic value, satisfy complex operational constraints, and remain causally grounded. Existing approaches lack a rigorous framework for meeting these requirements jointly. We address this gap by formulating NBA as a Constrained Markov Decision Process (CMDP). We then integrate off-policy causal correction of the estimated customer dynamics directly into the Linear Programming (LP) formulation of the CMDP. The resulting policy selects actions based on customer states to maximize long-term value while explicitly enforcing business guardrails. Furthermore, the LP formulation yields shadow prices that quantify the economic trade-offs of operational constraints (e.g., fairness and user welfare), enhancing policy interpretability. We validate our proposed offline Reinforcement Learning (RL) framework on an industry case study involving rideshare coupon targeting, where customer states are defined by Recency, Frequency, and Monetary (RFM) stages. We evaluate performance using model-based off-policy evaluation, with causal dynamics estimated from a temporal holdout. Under the observed coupon budget and RFM state representation, our optimized policy improves customer lifetime value (LTV) by 19% over the estimated behavior policy and 8% over a myopic baseline. Although the demonstrated action space is binary, the proposed framework is action-agnostic, yielding a general-purpose decision module for targeting systems that manage multiple interventions.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Markov decision process, Computer science, Reinforcement learning, Markov process, State space, Operations research

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange