The HeartBert model is introduced with three primary objectives: reducing the need for labeled data, minimizing computational resources, and simultaneously improving performance in machine learning systems that analyze Electrocardiogram (ECG) signals. Inspired by Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach, the HeartBert model-built on the RoBERTa architecture-generates sophisticated embeddings tailored for ECG-based projects in the medical domain. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to address complex challenges. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT, particularly in terms of its ability to perform well with smaller training datasets, reduced learning parameters, and effective performance compared to rival models. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.

翻译：HeartBERT模型旨在实现三个主要目标：减少对标注数据的需求、最小化计算资源，同时提升分析心电图（ECG）信号的机器学习系统性能。该模型受自然语言处理中双向编码器表示（BERT）的启发，并采用自监督学习方法进行增强，基于RoBERTa架构构建，可为医疗领域基于ECG的项目生成精细定制的嵌入表示。为展示所提出模型的通用性、可推广性和高效性，我们选取了两个关键下游任务：睡眠阶段检测与心跳分类。基于HeartBERT的系统采用双向LSTM头部设计，以应对复杂挑战。通过一系列实验验证了HeartBERT的优越性与先进性，特别是在较小训练数据集、较少学习参数条件下仍能保持良好性能，且相较于竞争模型具有更优表现。代码与数据已公开于https://github.com/ecgResearch/HeartBert。