Modern applications can generate a large amount of data from different sources with high velocity, a combination that is difficult to store and process via traditional tools. Hadoop is one framework that is used for the parallel processing of a large amount of data in a distributed environment, however, various challenges can lead to poor performance. Two particular issues that can limit performance are the high access time for I/O operations and the recomputation of intermediate data. The combination of these two issues can result in resource wastage. In recent years, there have been attempts to overcome these problems by using caching mechanisms. Due to cache space limitations, it is crucial to use this space efficiently and avoid cache pollution (the cache contains data that is not used in the future). We propose Hadoop-oriented SVM-LRU (HSVM- LRU) to improve Hadoop performance. For this purpose, we use an intelligent cache replacement algorithm, SVM-LRU, that combines the well-known LRU mechanism with a machine learning algorithm, SVM, to classify cached data into two groups based on their future usage. Experimental results show a significant decrease in execution time as a result of an increased cache hit ratio, leading to a positive impact on Hadoop performance.

翻译：现代应用程序能够以高速率从不同来源生成大量数据，这种组合难以通过传统工具存储和处理。Hadoop是一种用于在分布式环境中并行处理大数据的框架，然而，各种挑战可能导致性能不佳。两个特别限制性能的问题是I/O操作的高访问时间和中间数据的重新计算。这两个问题的结合可能导致资源浪费。近年来，已有尝试通过使用缓存机制来克服这些问题。由于缓存空间有限，高效利用此空间并避免缓存污染（缓存包含未来不再使用的数据）至关重要。我们提出面向Hadoop的SVM-LRU（HSVM-LRU）以提升Hadoop性能。为此，我们使用一种智能缓存替换算法SVM-LRU，该算法将著名的LRU机制与机器学习算法SVM相结合，根据未来使用情况将缓存数据分为两组。实验结果表明，由于缓存命中率的提高，执行时间显著减少，从而对Hadoop性能产生积极影响。