Newest papers

On Data Scaling in Tabular ML: More Data, Better Generalization?

Ivan Rubachev, Nikita Lazarev, Artem Babenko · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26794 · cited by 0 works (OpenAlex) · open-access PDF

Korean

그러한 테이블 기계 학습 데이터의 최근 작업은 다양하고 현실적인 평가 설정의 중요성을 강조했습니다. 대규모 (1M+) 데이터의 틈새는 여전히 탐구되지 않습니다. 훈련 데이터 스케일링을 연구하기 위해 우리는 8 개의 실제 테이블 데이터 세트를 최대 10 백만 개의 훈련 예제로 조립하여 작업의 자연 배포 변화를 유지하고 배포 변환 중에도 무작위 분할 평가를 사용합니다. 그러나 GBDT와 MLP 기반 모델은 광범위하게 유사한 규모 추세를 보여주며 추가 교육 데이터와 알고리즘 개선을 통해 이득을 얻습니다. 테이블 기초 모델은 파이프 조건에 대한 더 다양한 규모를 보여줍니다. 최근 버전은 배포 변수를 사용하여 더 큰 컨텍스트에서 더 일관되게 이

Machine translation (Meta M2M100); the English original follows.

Abstract

Recent work in tabular machine learning has emphasized the importance of diverse and realistic evaluation settings. The niche of large-scale (1M+) data, however, remains under-explored. To study training-data scaling, we assemble eight real-world tabular datasets with up to around ten million training examples, retaining the natural distribution shifts of the tasks and using random-split evaluation as a control. We find that GBDTs and MLP-based models show broadly similar scaling trends, with gains from both additional training data and algorithmic improvements. Tabular foundation models show more varied scaling behavior, with recent versions benefiting more consistently from larger contexts even under distribution shift. However, scaling training data under distribution shift introduces new challenges: in-distribution and out-of-distribution performance mostly improve together at smaller scales but can diverge at larger scales. We examine this through MLP capacity sweeps and comparisons of the validation distributions used for tuning and early stopping in MLPs and GBDTs. We also compare common practices of using validation data, contrasting its use for model selection in MLP pipelines with using it for in-context fitting in tabular foundation model pipelines. Under stronger shifts, MLPs selected using validation representative of the evaluation conditions generally maintain or improve performance as training data grows. In contrast, including such examples in context improves foundation-model performance, but does not prevent deterioration with increasing training size in our experiments.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Scaling, Computer science, Context (archaeology), Machine learning, Selection (genetic algorithm), Model selection

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange