Traditional data processing pipelines are typically static and handcrafted for specific tasks, limiting their adaptability to evolving requirements. While general-purpose agents and coding assistants can generate code for well-understood data pipelines, they lack the ability to autonomously monitor, manage, and optimize an end-to-end pipeline once deployed. We present \textbf{Autonomous Data Processing using Meta-agents} (ADP-MA), a framework that dynamically constructs, executes, and iteratively refines data processing pipelines through hierarchical agent orchestration. At its core, \textit{meta-agents} analyze input data and task specifications to design a multi-phase plan, instantiate specialized \textit{ground-level agents}, and continuously evaluate pipeline performance. The architecture comprises three key components: a planning module for strategy generation, an orchestration layer for agent coordination and tool integration, and a monitoring loop for iterative evaluation and backtracking. Unlike conventional approaches, ADP-MA emphasizes context-aware optimization, adaptive workload partitioning, and progressive sampling for scalability. Additionally, the framework leverages a diverse set of external tools and can reuse previously designed agents, reducing redundancy and accelerating pipeline construction. We demonstrate ADP-MA through an interactive demo that showcases pipeline construction, execution monitoring, and adaptive refinement across representative data processing tasks.

翻译：传统的数据处理流程通常是静态的，且针对特定任务手工设计，这限制了其对不断变化需求的适应性。虽然通用代理和编码助手可以为已知的数据流程生成代码，但它们缺乏在部署后自主监控、管理和优化端到端流程的能力。我们提出了**基于元代理的自主数据处理**框架，该框架通过分层代理编排，动态构建、执行并迭代优化数据处理流程。其核心在于，**元代理**分析输入数据和任务规范，设计多阶段计划，实例化专门的**底层代理**，并持续评估流程性能。该架构包含三个关键组件：用于策略生成的规划模块、用于代理协调与工具集成的编排层，以及用于迭代评估与回溯的监控循环。与传统方法不同，ADP-MA 强调上下文感知的优化、自适应的工作负载划分以及用于可扩展性的渐进采样。此外，该框架利用多样化的外部工具集，并能复用先前设计的代理，从而减少冗余并加速流程构建。我们通过一个交互式演示展示了 ADP-MA，该演示展示了在代表性数据处理任务中的流程构建、执行监控和自适应优化。