In-context learning (ICL) is the standard method for low-resource classification, yet its efficacy in specialized domains remains largely unexplored. We address the challenge of classifying semantically complex, multi-party B2B conversations, where traditional ICL encounters significant limitations, especially as context length increases due to the concatenation of multiple few-shot examples. We introduce the \texttt{Call Playbook} dataset, featuring five classification tasks derived from real-world B2B conversations targeting core sales concepts. To bridge the gap between performance and practical utility, we propose novel knowledge extraction methods that distill verbose examples into compact, interpretable representations of structured classification criteria and precise task descriptions. Our approach achieves a 99\% reduction in token usage and improves macro-averaged AUC by up to 7\% over traditional ICL. Notably, it remains robust as context grows, unlike advanced token compression baselines which degrade by over 9 F1 points. Importantly, our framework enables direct refinement of classification logic, addressing critical needs for transparency, efficiency, and user interaction in real-world NLP applications.

翻译：上下文学习（ICL）是低资源分类任务的标准方法，但其在专业领域中的有效性仍鲜有探索。我们针对语义复杂、多方参与的B2B对话分类挑战展开研究——此类场景中，传统ICL因多个少样本示例的拼接导致上下文长度增加，面临显著局限性。我们推出《通话剧本》数据集，该数据集包含基于真实B2B对话中核心销售概念的五个分类任务。为弥合性能与实用价值之间的差距，我们提出新颖的知识提取方法，将冗长示例提炼为结构化分类标准与精确任务描述的紧凑可解释表示。该方法在实现令牌使用量减少99%的同时，宏平均AUC较传统ICL提升最高达7%。值得注意的是，当上下文增长时，我们的方法仍保持稳健性，而先进的令牌压缩基线方法则会出现超过9个F1值的性能衰减。更重要的是，本框架支持对分类逻辑的直接优化，满足了实际NLP应用中对透明度、效率及人机交互的核心需求。