Extracting actionable suggestions from customer reviews is essential for operational decision-making, yet these directives are often embedded within mixed-intent, unstructured text. Existing approaches either classify suggestion-bearing sentences or generate high-level summaries, but rarely isolate the precise improvement instructions businesses need. We evaluate a hybrid pipeline combining a high-recall RoBERTa classifier trained with a precision-recall surrogate to reduce unrecoverable false negatives with a controlled, instruction-tuned LLM for suggestion extraction, categorization, clustering, and summarization. Across real-world hospitality and food datasets, the hybrid system outperforms prompt-only, rule-based, and classifier-only baselines in extraction accuracy and cluster coherence. Human evaluations further confirm that the resulting suggestions and summaries are clear, faithful, and interpretable. Overall, our results show that hybrid reasoning architectures achieve meaningful improvements fine-grained actionable suggestion mining while highlighting challenges in domain adaptation and efficient local deployment.
翻译:从客户评论中提取可执行建议对于运营决策至关重要,然而这些指令往往嵌入在混合意图、非结构化的文本中。现有方法要么对包含建议的句子进行分类,要么生成高层摘要,但很少能分离出企业所需的具体改进指令。我们评估了一种混合流程,该流程结合了一个高召回率的RoBERTa分类器(使用精确率-召回率替代目标训练以减少不可恢复的假阴性)和一个用于建议提取、分类、聚类与摘要的受控指令微调LLM。在真实的酒店和餐饮数据集上,该混合系统在提取准确性和聚类一致性方面均优于仅提示、基于规则以及仅分类器的基线方法。人工评估进一步证实,所生成的建议和摘要清晰、忠实且可解释。总体而言,我们的结果表明,混合推理架构在细粒度可执行建议挖掘方面实现了有意义的改进,同时也凸显了领域适应与高效本地部署方面的挑战。