Large language models show great potential in unstructured data understanding, but still face significant challenges with graphs due to their structural hallucination. Existing approaches mainly either verbalize graphs into natural language, which leads to excessive token consumption and scattered attention, or transform graphs into trainable continuous embeddings (i.e., soft prompt), but exhibit severe misalignment with original text tokens. To solve this problem, we propose to incorporate one special token <SOG_k> to fully represent the Structure Of Graph within a unified token space, facilitating explicit topology input and structural information sharing. Specifically, we propose a topology-aware structural tokenizer that maps each graph topology into a highly selective single token. Afterwards, we construct a set of hybrid structure Question-Answering corpora to align new structural tokens with existing text tokens. With this approach, <SOG_k> empowers LLMs to understand, generate, and reason in a concise and accurate manner. Extensive experiments on five graph-level benchmarks demonstrate the superiority of our method, achieving a performance improvement of 9.9% to 41.4% compared to the baselines while exhibiting interpretability and consistency. Furthermore, our method provides a flexible extension to node-level tasks, enabling both global and local structural understanding. The codebase is publicly available at https://github.com/Jingyao-Wu/SOG.

翻译：大型语言模型在非结构化数据理解方面展现出巨大潜力，但在处理图数据时，由于其结构幻觉问题，仍面临重大挑战。现有方法主要将图数据转化为自然语言描述，这会导致令牌消耗过多且注意力分散；或将图转化为可训练的连续嵌入（即软提示），但会与原始文本令牌产生严重错位。为解决此问题，我们提出引入一个特殊令牌<SOG_k>，在统一的令牌空间中完整表示图结构，从而促进显式拓扑输入和结构信息共享。具体而言，我们提出一种拓扑感知的结构令牌化器，将每个图拓扑映射为高度选择性的单个令牌。随后，我们构建了一套混合结构的问答语料库，以对齐新的结构令牌与现有文本令牌。通过这种方法，<SOG_k>使大语言模型能够以简洁准确的方式进行理解、生成和推理。在五个图级基准测试上的大量实验证明了我们方法的优越性，相较于基线方法实现了9.9%至41.4%的性能提升，同时展现出可解释性和一致性。此外，我们的方法可灵活扩展至节点级任务，支持全局和局部结构理解。代码库已公开于https://github.com/Jingyao-Wu/SOG。