The increasing demand for tabular data analysis calls for transitioning from manual architecture design to Neural Architecture Search (NAS). This transition demands an efficient and responsive anytime NAS approach that is capable of returning current optimal architectures within any given time budget while progressively enhancing architecture quality with increased budget allocation. However, the area of research on Anytime NAS for tabular data remains unexplored. To this end, we introduce ATLAS, the first anytime NAS approach tailored for tabular data. ATLAS introduces a novel two-phase filtering-and-refinement optimization scheme with joint optimization, combining the strengths of both paradigms of training-free and training-based architecture evaluation. Specifically, in the filtering phase, ATLAS employs a new zero-cost proxy specifically designed for tabular data to efficiently estimate the performance of candidate architectures, thereby obtaining a set of promising architectures. Subsequently, in the refinement phase, ATLAS leverages a fixed-budget search algorithm to schedule the training of the promising candidates, so as to accurately identify the optimal architecture. To jointly optimize the two phases for anytime NAS, we also devise a budget-aware coordinator that delivers high NAS performance within constraints. Experimental evaluations demonstrate that our ATLAS can obtain a good-performing architecture within any predefined time budget and return better architectures as and when a new time budget is made available. Overall, it reduces the search time on tabular data by up to 82.75x compared to existing NAS approaches.

翻译：表格数据分析需求的日益增长要求从手动架构设计向神经架构搜索（NAS）过渡。这一过渡需要一种高效且响应迅速的任意时刻NAS方法，该方法能够在任意给定时间预算内返回当前最优架构，同时随着预算分配的增加逐步提升架构质量。然而，针对表格数据的任意时刻NAS研究领域尚属空白。为此，我们提出ATLAS——首个专为表格数据设计的任意时刻NAS方法。ATLAS引入了一种新颖的两阶段过滤与精炼优化方案，结合无训练和基于训练的架构评估两种范式的优势。具体而言，在过滤阶段，ATLAS采用专为表格数据设计的新型零代价代理方法高效估计候选架构的性能，从而获得一组有前景的架构。随后在精炼阶段，ATLAS利用固定预算搜索算法调度有前景候选架构的训练，以准确识别最优架构。为实现两阶段在任意时刻NAS中的联合优化，我们还设计了预算感知协调器，在约束条件下提供高NAS性能。实验评估表明，我们的ATLAS可在任意预设时间预算内获得性能良好的架构，并在获得新时间预算时返回更优架构。总体而言，与现有NAS方法相比，它在表格数据上的搜索时间最多减少了82.75倍。