The increasing demand for tabular data analysis calls for transitioning from manual architecture design to Neural Architecture Search (NAS). This transition demands an efficient and responsive anytime NAS approach that is capable of returning current optimal architectures within any given time budget while progressively enhancing architecture quality with increased budget allocation. However, the area of research on Anytime NAS for tabular data remains unexplored. To this end, we introduce ATLAS, the first anytime NAS approach tailored for tabular data. ATLAS introduces a novel two-phase filtering-and-refinement optimization scheme with joint optimization, combining the strengths of both paradigms of training-free and training-based architecture evaluation. Specifically, in the filtering phase, ATLAS employs a new zero-cost proxy specifically designed for tabular data to efficiently estimate the performance of candidate architectures, thereby obtaining a set of promising architectures. Subsequently, in the refinement phase, ATLAS leverages a fixed-budget search algorithm to schedule the training of the promising candidates, so as to accurately identify the optimal architecture. To jointly optimize the two phases for anytime NAS, we also devise a budget-aware coordinator that delivers high NAS performance within constraints. Experimental evaluations demonstrate that our ATLAS can obtain a good-performing architecture within any predefined time budget and return better architectures as and when a new time budget is made available. Overall, it reduces the search time on tabular data by up to 82.75x compared to existing NAS approaches.

翻译：随着表格数据分析需求的日益增长，业界正从人工架构设计向神经架构搜索（NAS）过渡。这种过渡需要一种高效且响应及时的任意时间NAS方法，该方法能在任意给定时间预算内返回当前最优架构，同时随预算增加逐步提升架构质量。然而，针对表格数据的任意时间NAS研究领域仍属空白。为此，我们提出ATLAS——首个面向表格数据的任意时间NAS方法。ATLAS采用创新的两阶段过滤-精炼联合优化方案，融合了免训练与基于训练的架构评估范式的优势。具体而言，在过滤阶段，ATLAS使用专为表格数据设计的新型零成本代理架构，高效评估候选架构性能，从而获得一组有潜力的架构。随后在精炼阶段，ATLAS采用固定预算搜索算法调度有潜力候选架构的训练，以精确识别最优架构。为联合优化两阶段实现任意时间NAS，我们还设计了预算感知协调器，能在约束条件下实现高NAS性能。实验评估表明，ATLAS能在任意预设时间预算内获得性能优异的架构，并在获得新时间预算时返回更优架构。总体而言，与现有NAS方法相比，该方法将表格数据的搜索时间最高缩短82.75倍。