The number of IoT devices is expected to continue its dramatic growth in the coming years and, with it, a growth in the amount of data to be transmitted, processed and stored. Compression techniques that support analytics directly on the compressed data could pave the way for systems to scale efficiently to these growing demands. This paper proposes two novel methods for preprocessing a stream of floating point data to improve the compression capabilities of various IoT data compressors. In particular, these techniques are shown to be helpful with recent compressors that allow for random access and analytics while maintaining good compression. Our techniques improve compression with reductions up to 80% when allowing for at most 1% of recovery error.
翻译:物联网设备数量预计在未来几年将持续快速增长,随之而来的是需要传输、处理和存储的数据量的增长。支持直接对压缩数据进行分析的压缩技术,可以为系统高效扩展以满足这些日益增长的需求铺平道路。本文提出了两种新颖的浮点数据流预处理方法,以提升各种物联网数据压缩器的压缩能力。具体而言,这些技术被证明对最近允许随机访问和分析同时保持良好压缩性能的压缩器尤为有效。当允许最多1%的恢复误差时,我们的技术可实现高达80%的压缩率提升。