New DataFlow 2.0 is now available

Standardize Data. Visualize Value.

The visual ETL platform that bridges the gap between chaos and clarity. Build complex Map-Reduce pipelines with simple visuals and powerful Python code.

Trusted by 12,000+ data teams

ACME Corp StarkInd Wayne Ent Cyberdyne Massive Dynamic

Everything you need to ship pipeline.

Stop wrestling with airflow configurations. Start shipping value.

Visual Sub-Pipelines

Abstract away complexity. Nest flows within flows using Map nodes to process data lists efficiently. Manage scale with ease.

Map-Reduce Nested Flows Iterators

Full Python Power

Don't limit yourself to GUI. Write full Pandas transformations and aggregations directly in Reduce nodes.

Latency 24ms

AI-Native Design

Chat with Gemini to generate complete pipelines in seconds. From 'Idea' to 'Pipeline' instantly.

Enterprise Grade Security

SOC2 Type II ready. Role-based access control.
Your data never leaves your VPC.

policy_check: passed
encryption: enabled (AES-256)
access_log: active
// Audited 2 mins ago

Code when you want.
Visual when you need.

DataFlow gives you the best of both worlds. Build complex logic in Python or SQL, wrap it in a reusable node, and let your analysts connect the dots.

  • Git-backed version control
  • Full Python/Pydantic support
  • dbt Core integration built-in
transform.py
def transform(df):
    # Full DataFrame Power
    if df.empty: return df
    
    # Group By and Aggregate
    result = df.groupby('currency').agg({
        'latest_close': 'sum',
        'ticker': 'count'
    }).reset_index()
    
    return result

Start in 5 minutes.

$ pip install dataflow-core
$ dataflow init my-project
$ dataflow up # Launches UI on localhost:3000