Newest papers

Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

Skander Lejmi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26789 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Un creador de mercado está configurado para proporcionar liquidez mediante la cotización de precios establecidos para comprar y vender un activo, ganando el spread a la vez que gestiona el riesgo de inventario. Estudiamos a los creadores de mercado capacitados con aprendizaje de reforzamiento de varios agentes en los mercados de predicción, específicamente Polymarket, cuyos libros de orden delgado presentan desafíos distintos. Tres desafíos apoyan esta investigación: la selección adversa puede causar una política entrenada en datos reales para detener la cotización; las consecuencias de una cotización pueden llegar a muchos pasos después de la acción; y los datos del mercado de predicción-valor reciente L2 no identifican las órdenes individuales necesarias para la simulación. Nosotros abordamos estos desafíos positivos reconstruyendo los datos del formato LOBST

Machine translation (Meta M2M100); the English original follows.

Abstract

A market maker provides liquidity by quoting prices to buy and sell an asset, earning the spread while managing inventory risk. We study market makers trained with multi-agent reinforcement learning on prediction markets, specifically Polymarket, whose thin order books pose distinct challenges. Three challenges underpin this research: adverse selection may cause a policy trained on real data to stop quoting; the consequences of a quote can arrive many steps after the action; and prediction-market L2 data does not identify individual orders needed for simulation. We address these challenges by reconstructing order-level LOBSTER-format data from Polymarket and training agents in JaxMARL-HFT on 417 real markets. We enrich the baseline market maker's observation space with the book and recent flow, redesign its actions and inventory controls, and propose a cooperative multi-agent decomposition into a quoter and a risk manager trained using recurrent Independent Proximal Policy Optimisation (IPPO) with RNN policies. We investigate reward shaping, particularly reward blending and post-fill markout, to mitigate credit-assignment difficulties between the agents. Our redesigned single-agent market maker quotes more frequently and incurs smaller held-out losses than the retrained JaxMARL-HFT baseline, although the contributions of jointly changed observations, actions and risk constraints are not separately identified. Decomposition alone performs poorly; careful reward design substantially improves the desk relative to the unshaped split and yields our best result. Models are evaluated on realised end-of-episode portfolio value on held-out markets. None establishes positive average portfolio value in periodic held-out evaluation, and each configuration was trained with one seed. The environment, data and protocol lay a foundation for makers that quote many linked markets at once.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Portfolio, Desk, Computer science, Order (exchange), Market maker

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange