Arjun Ashok, Mihir Parmar, Palash Goyal, Bhavana Dalvi Mishra, Chunliang Li, Irina Rish, Alexandre Drouin, Jinsung Yoon, Tomas Pfister · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26582 · cited by 0 works (OpenAlex) · open-access PDF
Los modelos de base de la serie de novelas recientes (TSFMs) pueden consumir covariantes en el contexto, extendiendo la predicción de disparo cero a problemas impulsados por factores externos. Esto permite el análisis de escenarios: predecir cómo un objetivo responde a eventos que establecen covariantes a valores futuros elegidos. Si pueden hacer esto de forma fiable es una pregunta abierta en el contexto. Introducimos WhatIf-Bench, un punto de referencia para medir cuán bien los predicadores utilizan covariantes para predecir la respuesta de un objetivo a eventos futuros. WhatIf-Bench es semi-sintético: las series de tiempo provienen de simuladores mecánicos de centros de datos, redes de energía, tráfico por carretera y redes de agua. Introducimos WhatIf-Bench, un marco de referencia para medir
Machine translation (Meta M2M100); the English original follows.
Recent time series foundation models (TSFMs) can consume covariates in-context, extending zero-shot forecasting to problems driven by external factors. This enables scenario analysis: forecasting how a target responds to events that set covariates to chosen future values. Whether they can do this reliably is an open question. We introduce WhatIf-Bench, a benchmark measuring how well forecasters use covariates to forecast a target's response to future events. WhatIf-Bench is semi-synthetic: time series come from mechanistic simulators of data centers, power grids, road traffic, and water networks, with controlled covariate-target relationships grounded in each system. Each task presents a history of events intervening on covariates, then asks to forecast a future event that either replays seen values (Replay) or sets them to unseen ones (Novel). Varying the number of covariates, historical events, and event length yields 37,665 tasks. Evaluating TSFMs, frontier LLMs, and simple regression baselines, we find the LLMs reach the highest skill (improvement over a naive baseline), ahead of the best TSFMs. Every model forecasts well on Replay futures (48 to 85% skill) but drops to -7 to 50% on Novel futures, with the regression baselines falling the most. Skill on Novel futures also falls as events set more covariates jointly. More historical events raise skill on Novel futures across the board, but large differences remain. WhatIf-Bench reveals generalization to Novel futures as the core challenge, and offers a testbed toward arbitrary scenario analysis.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Futures contract, Benchmarking, Benchmark (surveying), Computer science, Time series, Covariate
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26582, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange