Arjun Ashok, Mihir Parmar, Palash Goyal, Bhavana Dalvi Mishra, Chunliang Li, Irina Rish, Alexandre Drouin, Jinsung Yoon, Tomas Pfister · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26582 · cited by 0 works (OpenAlex) · open-access PDF
Novarys baru-baru ini mengungkapkan model dasar seri inti (TSFMs) yang dapat mengkonsumsi covariates dalam konteks, memperluas prediksi nol-shot ke masalah yang didorong oleh faktor eksternal. Ini memungkinkan analisis skenario: memprediksi bagaimana target merespon peristiwa yang menetapkan covariates untuk nilai masa depan yang dipilih. Apakah mereka dapat melakukan ini secara dapat diandalkan adalah pertanyaan terbuka pada TS-Bench, yang memperkenalkan WhatIf-Bench, benchmark yang mengukur seberapa baik prediksi menggunakan covariates untuk memprediksi tanggapan target terhadap peristiwa masa depan. WhatIf-Bench adalah semi-synthetic: seri waktu berasal dari simulator mekanis dari pusat data, grid daya, lalu lintas jalan, dan jaringan air. Kami memperkenalkan WhatIf-Bench, dengan hubungan futures covariate
Machine translation (Meta M2M100); the English original follows.
Recent time series foundation models (TSFMs) can consume covariates in-context, extending zero-shot forecasting to problems driven by external factors. This enables scenario analysis: forecasting how a target responds to events that set covariates to chosen future values. Whether they can do this reliably is an open question. We introduce WhatIf-Bench, a benchmark measuring how well forecasters use covariates to forecast a target's response to future events. WhatIf-Bench is semi-synthetic: time series come from mechanistic simulators of data centers, power grids, road traffic, and water networks, with controlled covariate-target relationships grounded in each system. Each task presents a history of events intervening on covariates, then asks to forecast a future event that either replays seen values (Replay) or sets them to unseen ones (Novel). Varying the number of covariates, historical events, and event length yields 37,665 tasks. Evaluating TSFMs, frontier LLMs, and simple regression baselines, we find the LLMs reach the highest skill (improvement over a naive baseline), ahead of the best TSFMs. Every model forecasts well on Replay futures (48 to 85% skill) but drops to -7 to 50% on Novel futures, with the regression baselines falling the most. Skill on Novel futures also falls as events set more covariates jointly. More historical events raise skill on Novel futures across the board, but large differences remain. WhatIf-Bench reveals generalization to Novel futures as the core challenge, and offers a testbed toward arbitrary scenario analysis.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Futures contract, Benchmarking, Benchmark (surveying), Computer science, Time series, Covariate
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26582, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange