Self-supervised learning offers an efficient way of extracting rich representations from various types of unlabeled data while avoiding the cost of annotating large-scale datasets. This is achievable by designing a pretext task to form pseudo labels with respect to the modality and domain of the data. Given the evolving applications of online handwritten texts, in this study, we propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages, along with two suggested pipelines for fine-tuning the pretrained models. To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods. The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification, also highlighting the superiority of utilizing the pretrained models over the models trained from scratch.

翻译：自监督学习提供了一种高效的方法，从各种未标记数据中提取丰富表示，同时避免了大规模数据集标注的成本。这通过设计一个自监督任务（pretext task）来实现，该任务根据数据的模态和领域形成伪标签。鉴于在线手写文本应用不断发展，本研究提出了一种新颖的“部分笔迹掩码”（Part of Stroke Masking, POSM）作为自监督任务，用于预训练模型，以从英语和汉语的个体在线手写中提取信息性表示，并提出了两种用于微调预训练模型的流程。为了评估提取表示的质量，我们采用了内在评估和外在评估两种方法。预训练模型经过微调，在作者识别、性别分类和利手分类等任务中取得了当前最优结果，这也突显了利用预训练模型相较于从头训练模型的优越性。