Learning to represent free text is a core task in many clinical machine learning (ML) applications, as clinical text contains observations and plans not otherwise available for inference. State-of-the-art methods use large language models developed with immense computational resources and training data; however, applying these models is challenging because of the highly varying syntax and vocabulary in clinical free text. Structured information such as International Classification of Disease (ICD) codes often succinctly abstracts the most important facts of a clinical encounter and yields good performance, but is often not as available as clinical text in real-world scenarios. We propose a \textbf{multi-view learning framework} that jointly learns from codes and text to combine the availability and forward-looking nature of text and better performance of ICD codes. The learned text embeddings can be used as inputs to predictive algorithms independent of the ICD codes during inference. Our approach uses a Graph Neural Network (GNN) to process ICD codes, and Bi-LSTM to process text. We apply Deep Canonical Correlation Analysis (DCCA) to enforce the two views to learn a similar representation of each patient. In experiments using planned surgical procedure text, our model outperforms BERT models fine-tuned to clinical data, and in experiments using diverse text in MIMIC-III, our model is competitive to a fine-tuned BERT at a tiny fraction of its computational effort.

翻译：学习表示自由文本是许多临床机器学习（ML）应用中的核心任务，因为临床文本包含观察和计划等信息，而这些信息在其他方式下无法用于推理。最先进的方法使用了通过巨大计算资源和训练数据开发的大语言模型；然而，由于临床自由文本中高度变化的句法和词汇，应用这些模型具有挑战性。诸如国际疾病分类（ICD）代码等结构化信息通常简洁地概括了临床接触中最重要的信息，并产生良好的性能，但在现实场景中往往不如临床文本那样容易获取。我们提出了一种**多视图学习框架**，它联合从代码和文本中学习，以结合文本的可用性和前瞻性以及ICD代码的更好性能。学习到的文本嵌入可以在推理过程中作为预测算法的输入，而独立于ICD代码。我们的方法使用图神经网络（GNN）处理ICD代码，使用Bi-LSTM处理文本。我们应用深度典型相关分析（DCCA）来强制两个视图学习每个患者的相似表示。在使用计划手术程序文本的实验中，我们的模型优于在临床数据上微调的BERT模型；在使用MIMIC-III中多样化文本的实验中，我们的模型以极小的计算代价达到与微调BERT相当的性能。