It is a sad reflection of modern academia that code is often ignored after publication -- there is no academic 'kudos' for bug fixes / maintenance. Code is often unavailable or, if available, contains bugs, is incomplete, or relies on out-of-date / unavailable libraries. This has a significant impact on reproducibility and general scientific progress. Neural Architecture Search (NAS) is no exception to this, with some prior work in reproducibility. However, we argue that these do not consider long-term reproducibility issues. We therefore propose a checklist for long-term NAS reproducibility. We evaluate our checklist against common NAS approaches along with proposing how we can retrospectively make these approaches more long-term reproducible.

翻译：现代学术界悲哀地看到,代码在出版后常常被忽略 -- -- 没有用于错误修正/维护的学术“kudos” 。 代码经常没有,或者如果有的话,含有错误,不完整,或者依赖过时/没有的图书馆。 这对可复制性和总体科学进步有重大影响。 神经结构搜索(NAS)也不例外,此前还做了一些可复制性工作。 然而,我们认为,这些并不考虑长期的可复制性问题。 因此,我们提出了一个长期NAS可复制性的核对表。 我们对照普通的NAS方法评估我们的核对表,同时建议我们如何使这些方法可以追溯性地更长期地复制。