Pitch is a foundational aspect of our perception of audio signals. Pitch contours are commonly used to analyze speech and music signals and as input features for many audio tasks, including music transcription, singing voice synthesis, and prosody editing. In this paper, we describe a set of techniques for improving the accuracy of widely-used neural pitch and periodicity estimators to achieve state-of-the-art performance on both speech and music. We also introduce a novel entropy-based method for extracting periodicity and per-frame voiced-unvoiced classifications from statistical inference-based pitch estimators (e.g., neural networks), and show how to train a neural pitch estimator to simultaneously handle both speech and music data (i.e., cross-domain estimation) without performance degradation. While neural pitch trackers have historically been significantly slower than signal processing based pitch trackers, our estimator implementations approach the speed of state-of-the-art DSP-based pitch estimators on a standard CPU, but with significantly more accurate pitch and periodicity estimation. Our experiments show that an accurate, cross-domain pitch and periodicity estimator written in PyTorch with a hopsize of ten milliseconds can run 11.2x faster than real-time on a Intel i9-9820X 10-core 3.30 GHz CPU or 408x faster than real-time on a NVIDIA GeForce RTX 3090 GPU, without hardware optimization. We release all of our code and models as Pitch-Estimating Neural Networks (penn), an open-source, pip-installable Python module for training, evaluating, and performing inference with pitch- and periodicity-estimating neural networks. The code for penn is available at https://github.com/interactiveaudiolab/penn.

翻译：音高是人类感知音频信号的基础维度。音高轮廓通常用于分析语音和音乐信号，并作为众多音频任务（包括音乐转谱、歌声合成及韵律编辑）的输入特征。本文描述了一系列技术方法，旨在提升广泛使用的神经音高与周期性估计器的准确率，使其在语音和音乐领域均达到当前最佳性能。我们同时提出了一种基于熵的新方法，用于从基于统计推断的音高估计器（如神经网络）中提取周期性和逐帧有声/无声分类，并展示了如何训练一个神经音高估计器，使其在性能不衰减的前提下同时处理语音与音乐数据（即跨域估计）。尽管神经音高追踪器历来比基于信号处理的音高追踪器慢得多，但我们的估计器实现在标准CPU上的运行速度已接近基于数字信号处理的当前最优音高估计器，同时具备更高的音高与周期性估计精度。实验表明，一个基于PyTorch编写、跳步长度为十毫秒的精准跨域音高与周期性估计器，在Intel i9-9820X 10核3.30 GHz CPU上可达到实时运行速度的11.2倍，或在NVIDIA GeForce RTX 3090 GPU上达到实时运行速度的408倍（无需硬件优化）。我们将所有代码和模型以Pitch-Estimating Neural Networks (penn) 名义开源发布，这是一个可通过pip安装的Python模块，支持音高与周期性估计神经网络的训练、评估和推理。penn的代码可在https://github.com/interactiveaudiolab/penn获取。