git404hub

what is datax fr?

alibaba/datax — explained in plain English

Analysis updated 2026-06-24

★ 17,197JavaAudience · dataComplexity · 3/5Setup · moderate

tl;dr

Alibaba's open-source batch data synchronization framework that moves data between databases, warehouses, and storage systems using a plugin architecture where each source needs only a Reader and Writer plugin.

vibe map

mindmap
  root((DataX))
    What it does
      Batch data sync
      Plugin framework
      Cross-system ETL
    Sources supported
      MySQL Oracle PostgreSQL
      HDFS Hive HBase
      Alibaba Cloud OSS
      Neo4j GDB
    Use cases
      Database migration
      Warehouse loading
      Multi-source consolidation
    Tech stack
      Java
      Plugin architecture

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Migrate an entire MySQL database to a data warehouse like Hive or MaxCompute in one job.

VIBE 2

Set up a recurring batch pipeline to sync operational data from PostgreSQL into HDFS.

VIBE 3

Pull data from multiple sources such as Oracle, SQL Server, and HBase into a single analysis environment.

VIBE 4

Move data between Alibaba Cloud services like OSS and DataHub without writing custom ETL code.

what's the stack?

Java

how it stacks up fr

alibaba/dataxfacebook/frescomaterial-components/material-components-android
Stars17,19717,15017,280
LanguageJavaJavaJava
Setup difficultymoderatemoderatemoderate
Complexity3/53/53/5
Audiencedatadeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · moderate time til it works · 30min

Requires Java and DataX configuration JSON files for each Reader and Writer plugin pair.

License not mentioned in the explanation.

in plain english

DataX is an open-source data synchronisation tool from Alibaba. It is the open-source version of Alibaba Cloud's DataWorks data integration service and is widely used inside Alibaba for moving data between different storage systems in bulk. The job it solves is the common one of copying tables, files or events from one database or storage system to another, especially when the source and destination speak different protocols. The design is plugin-based. DataX itself is a synchronisation framework, each kind of data source is supported by a pair of plugins, a Reader that pulls data out of the source and a Writer that pushes data into the destination. Because every connector is just a plugin, adding support for a new data source automatically makes it interoperable with all the existing ones, without having to write a special pipeline for each pair. Out of the box, DataX ships with plugins for relational databases like MySQL, Oracle, OceanBase, SQL Server, PostgreSQL, DRDS and Kingbase, for big-data and warehouse systems like HDFS, Hive, HBase, MaxCompute (ODPS), Hologres and ADS, for Alibaba Cloud storage and middleware such as OSS, OCS, DataHub and SLS, for graph databases like Alibaba's GDB and Neo4j, and for newer engines such as databend. You would actually use DataX if you need to do a one-off migration of a whole database, set up a recurring batch upload of operational data into a data warehouse, or pull data from many different systems into a single analysis environment. DataX is written in Java, and a commercial managed version with real-time capabilities is offered separately as part of Alibaba Cloud DataWorks. The full README is longer than what was provided.

prompts (copy fr)

prompt 1
Show me how to configure a DataX job to copy a table from MySQL to PostgreSQL, including the Reader and Writer plugin JSON config.
prompt 2
How do I write a custom DataX plugin to add support for a new database as a Reader?
prompt 3
Give me a DataX job config that reads from HDFS and writes to HBase with column mapping.
prompt 4
How do I schedule a DataX sync job to run nightly using a cron job on Linux?
prompt 5
What DataX Reader and Writer combination should I use to move data from Oracle into Alibaba Cloud MaxCompute?

Frequently asked questions

what is datax fr?

Alibaba's open-source batch data synchronization framework that moves data between databases, warehouses, and storage systems using a plugin architecture where each source needs only a Reader and Writer plugin.

What language is datax written in?

Mainly Java. The stack also includes Java.

What license does datax use?

License not mentioned in the explanation.

How hard is datax to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is datax for?

Mainly data.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.

similar repos, if this ain't it