Newest papers

Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

Skander Lejmi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26789 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Um criador de mercado foi configurado para fornecer liquidez por preços estabelecidos para comprar e vender um ativo, ganhando o spread enquanto gerenciando o risco de inventário. Estudamos os criadores de mercado treinados com reforço de multi-agente de aprendizagem em mercados de previsão, especificamente Polymarket, cujos livros de ordem fina apresentam desafios distintos. Três desafios sustentam esta pesquisa: a seleção adversa pode causar uma política treinada em dados reais para parar de citar; as consequências de uma cotação podem chegar a muitos passos após a ação; e os dados de predição-mercado L2 não identificam as ordens individuais necessárias para a simulação. Nós abordamos esses desafios positivos reconstruindo os dados do formato LOBSTER de nível de ordem de

Machine translation (Meta M2M100); the English original follows.

Abstract

A market maker provides liquidity by quoting prices to buy and sell an asset, earning the spread while managing inventory risk. We study market makers trained with multi-agent reinforcement learning on prediction markets, specifically Polymarket, whose thin order books pose distinct challenges. Three challenges underpin this research: adverse selection may cause a policy trained on real data to stop quoting; the consequences of a quote can arrive many steps after the action; and prediction-market L2 data does not identify individual orders needed for simulation. We address these challenges by reconstructing order-level LOBSTER-format data from Polymarket and training agents in JaxMARL-HFT on 417 real markets. We enrich the baseline market maker's observation space with the book and recent flow, redesign its actions and inventory controls, and propose a cooperative multi-agent decomposition into a quoter and a risk manager trained using recurrent Independent Proximal Policy Optimisation (IPPO) with RNN policies. We investigate reward shaping, particularly reward blending and post-fill markout, to mitigate credit-assignment difficulties between the agents. Our redesigned single-agent market maker quotes more frequently and incurs smaller held-out losses than the retrained JaxMARL-HFT baseline, although the contributions of jointly changed observations, actions and risk constraints are not separately identified. Decomposition alone performs poorly; careful reward design substantially improves the desk relative to the unshaped split and yields our best result. Models are evaluated on realised end-of-episode portfolio value on held-out markets. None establishes positive average portfolio value in periodic held-out evaluation, and each configuration was trained with one seed. The environment, data and protocol lay a foundation for makers that quote many linked markets at once.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Portfolio, Desk, Computer science, Order (exchange), Market maker

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange