A churn model is useful through the customers it selects for a retention offer: the better model is the one whose selection makes the campaign more profitable. Tabular foundation models (TFMs), pretrained networks that predict from a table of labelled examples without being trained on it, were recently reported to decisively outperform tree ensembles for churn. That comparison, however, used untuned gradient-boosted trees (GBDTs) and ranking metrics only. We ask whether the advantage holds against tuned GBDTs when models are judged by expected campaign profit and by the reliability of their probabilities. Four TFMs (TabPFN-2.5, TabPFN-3.5, TabPFN-3.5-Fast, TabICL v2) face LightGBM, XGBoost and CatBoost tuned with Optuna on validation AUC, on five public telecom datasets, three from real operators, with 10 repeated splits each and 20 pre-declared comparisons tested with a corrected t-test and Holm correction. TabPFN-3.5 earns significantly more expected maximum profit (EMP) than each tuned GBDT on two datasets, Cell2Cell (1.9%) and Iranian (7.6%). On Orange it is ahead in every split (14.3%), but not significantly after correction, and no TFM differs significantly from the GBDTs on the two fictional datasets. Across 27 assumed combinations of customer value, incentive cost and acceptance rate, TabPFN-3.5 stays ahead on all three real-operator datasets, though the size of its gain varies widely. Retuning the GBDTs on EMP moved no gap by more than 0.014 per customer. Ranking and probability reliability diverge: raw TabICL v2 underpredicts churn on Orange, and a threshold policy built on its probabilities loses 15.7% of attainable profit until it is recalibrated. Because the splits resample the same customers, the results describe these datasets, not telecom operators in general. The non-commercial licence of the TabPFN weights, rather than scoring cost, restricts production use.
翻译:暂无翻译