Newest papers

Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

Skander Lejmi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26789 · cited by 0 works (OpenAlex) · open-access PDF

Russian

Мы изучаем маркетологов, обученных с помощью многоагентного усиления обучения на предсказательных рынках, в частности Polymarket, чьи книги тонкого порядка представляют собой различные вызовы. Три вызова поддерживают это исследование: неблагоприятный отбор агентов в JaxMARL-HFT может вызвать политику, обученную на реальных данных, чтобы остановить котировки; последствия котировки могут достичь многих шагов после действия; и данные прогнозирования рынка L2 не идентифицируют отдельные заказы, необходимые для симуляции. Мы решаем эти положительные вызовы, реконструируя данные на уровне порядка LOBSTER-формата от Polymarket и тренируя агентов в JaxMARL-HFT на 417 реальных рынках. Мы обогаща

Machine translation (Meta M2M100); the English original follows.

Abstract

A market maker provides liquidity by quoting prices to buy and sell an asset, earning the spread while managing inventory risk. We study market makers trained with multi-agent reinforcement learning on prediction markets, specifically Polymarket, whose thin order books pose distinct challenges. Three challenges underpin this research: adverse selection may cause a policy trained on real data to stop quoting; the consequences of a quote can arrive many steps after the action; and prediction-market L2 data does not identify individual orders needed for simulation. We address these challenges by reconstructing order-level LOBSTER-format data from Polymarket and training agents in JaxMARL-HFT on 417 real markets. We enrich the baseline market maker's observation space with the book and recent flow, redesign its actions and inventory controls, and propose a cooperative multi-agent decomposition into a quoter and a risk manager trained using recurrent Independent Proximal Policy Optimisation (IPPO) with RNN policies. We investigate reward shaping, particularly reward blending and post-fill markout, to mitigate credit-assignment difficulties between the agents. Our redesigned single-agent market maker quotes more frequently and incurs smaller held-out losses than the retrained JaxMARL-HFT baseline, although the contributions of jointly changed observations, actions and risk constraints are not separately identified. Decomposition alone performs poorly; careful reward design substantially improves the desk relative to the unshaped split and yields our best result. Models are evaluated on realised end-of-episode portfolio value on held-out markets. None establishes positive average portfolio value in periodic held-out evaluation, and each configuration was trained with one seed. The environment, data and protocol lay a foundation for makers that quote many linked markets at once.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Portfolio, Desk, Computer science, Order (exchange), Market maker

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange