Traditional data processing pipelines are typically static and handcrafted for specific tasks, limiting their adaptability to evolving requirements. While general-purpose agents and coding assistants can generate code for well-understood data pipelines, they lack the ability to autonomously monitor, manage, and optimize an end-to-end pipeline once deployed. We present \textbf{Autonomous Data Processing using Meta-agents} (ADP-MA), a framework that dynamically constructs, executes, and iteratively refines data processing pipelines through hierarchical agent orchestration. At its core, \textit{meta-agents} analyze input data and task specifications to design a multi-phase plan, instantiate specialized \textit{ground-level agents}, and continuously evaluate pipeline performance. The architecture comprises three key components: a planning module for strategy generation, an orchestration layer for agent coordination and tool integration, and a monitoring loop for iterative evaluation and backtracking. Unlike conventional approaches, ADP-MA emphasizes context-aware optimization, adaptive workload partitioning, and progressive sampling for scalability. Additionally, the framework leverages a diverse set of external tools and can reuse previously designed agents, reducing redundancy and accelerating pipeline construction. We demonstrate ADP-MA through an interactive demo that showcases pipeline construction, execution monitoring, and adaptive refinement across representative data processing tasks.

翻译：传统的数据处理流程通常是静态的，且针对特定任务手工设计，这限制了其适应不断变化需求的能力。虽然通用智能体和编码助手可以为已充分理解的数据流程生成代码，但它们缺乏在部署后自主监控、管理和优化端到端流程的能力。本文提出 **基于元智能体的自主数据处理**（ADP-MA），这是一个通过分层智能体编排动态构建、执行并迭代优化数据处理流程的框架。其核心在于，**元智能体**通过分析输入数据和任务规范来设计多阶段计划，实例化专门的**底层智能体**，并持续评估流程性能。该架构包含三个关键组件：用于策略生成的规划模块、用于智能体协调与工具集成的编排层，以及用于迭代评估与回溯的监控循环。与传统方法不同，ADP-MA 强调上下文感知的优化、自适应的工作负载划分以及用于可扩展性的渐进采样。此外，该框架利用了多样化的外部工具集，并能复用先前设计的智能体，从而减少冗余并加速流程构建。我们通过一个交互式演示展示了 ADP-MA，该演示在典型的数据处理任务中展现了流程构建、执行监控与自适应优化。