Mechanism-first data environments

Scaling Data Environmentsfrom Scratch

Specify the world behind the files.
Generate realistic data. Ground analytical supervision.

Zhanli Li^{3,4 † §}Zeyuan Xu⁵Chaoxu Pang³Zeyuan Liu³An-Yang Ji^{1,2,3 †}Te Sun⁶ Kongcheng Zhang^{3,7 †}Wenbo Xu^{3,8 †}Xubin Yue^{3,7 †}Shipeng Luo⁹ Junjie Zhao⁶Yingfan Hua¹⁰Jingqing Ruan³Zhongda Su³Feng Hong^{3 *} Zhizhao Zeng³Ke Zeng³Xunliang Cai³Han-Jia Ye^{1,2 *}

Longcat Interaction Team, Meituan · Nanjing University · and collaborators

All affiliations & author notes

¹ School of Artificial Intelligence, Nanjing University

² National Key Laboratory for Novel Software Technology, Nanjing University

³ Longcat Interaction Team, Meituan

⁴ Zhongnan University of Economics and Law

⁵ Hunan University

⁶ Shanghai Jiao Tong University

⁷ Zhejiang University

⁸ Nanjing University

⁹ Institute of Automation, Chinese Academy of Sciences

¹⁰ The Chinese University of Hong Kong

† Work done during an internship at Longcat Interaction Team, Meituan. § First author. * Corresponding authors.

Code

Paper

Collection

Code, the paper PDF, and the Hugging Face collection are available.

01 / The idea

Build the world.
Not just the tables.

Real data is costly to curate, difficult to share, and often missing the generative truth needed to verify open-ended findings. Exploring files alone can also leave valid insights without matching evaluation criteria.

Data World reverses the process. We specify mechanisms first, construct a coherent canonical state, and project it into realistic, imperfect files. Independent audits check whether selected analytical targets remain recoverable from those observations.

Known mechanisms.
Imperfect observations.
Grounded supervision.

DAG → ENV → TASK

DAG → layer-wise environment generation → task synthesis.

Example synthesized taskWhat explains the rise in late deliveries, and how does the effect vary across warehouses?

Illustrative animation, not a live generation run. The solver sees the environment and task—not generation records or rubrics.

02 / The engine

From mechanisms
to measurable insight.

Independent agent sessions communicate through artifacts. A dependency DAG guides generation; a generative record grounds tasks and rubrics.

Data World pipeline: world specification, layer-wise canonical state construction, observation projection, independent audit, task synthesis, solving and evaluation — The complete Data World synthesis and evaluation pipeline.Click to explore ↗

01 — SPECIFY

World contract

Define entities, timelines, edge mechanisms, invariants, and observation systems before generating any files.

02 — CONSTRUCT

Canonical state

Build topological layers in order, using committed upstream state as read-only input.

03 — OBSERVE

Realistic imperfections

Project truth through explicit missingness, delays, identifier mismatches, duplicates, and drift.

04 — VERIFY

Audit & supervision

Audit relationships and target recoverability, then generate tasks with atomic positive and negative rubric items.

03 / Evaluation

Beyond getting
the numbers right.

DataWorld-Bench separates data computation from insight discovery. Strong processing does not automatically yield grounded analytical conclusions.

DataWorld-Bench · model comparison

Each model has a consistent color; shading and patterns distinguish the three scores. Higher is better.

Order by

Data computationInsight discoveryAverage

Hover, focus, or tap a bar to see its exact mean and standard deviation.

Manuscript-reported normalized rubric scores × 100. Colors identify models: light fill = data computation, diagonal stripes = insight discovery, solid fill = average. Bars show means, not stacked scores; SD is available on each bar and in the full table. Average pools all rubric items and need not equal the mean of the two section scores. On smaller screens, swipe horizontally to view all models.

Every model scores lower on insight discovery than on data computation—a gap of 4.87–21.35 points.

View full results · scores, standard deviations & interaction costs

Mean (SD). Score columns are normalized rubric scores × 100; bold / underlined means denote best / second-best scores. Rounds count model calls. Total tokens are prompt plus output of the final call; response tokens sum outputs over the run. Token columns are in thousands.
Model	Score ↑			Interaction and length
Model	Data computation	Insight discovery	Average	Rounds	Total tokens (k)	Resp. tokens (k)
GPT-6-Astra	87.80 (16.38)	66.45 (20.98)	74.43 (15.85)	28.7 (8.7)	121.9 (36.4)	88.8 (18.9)
Qwen-3.8-Max-0902	82.36 (17.79)	67.95 (22.92)	73.43 (16.81)	82.7 (23.5)	289.7 (81.9)	125.8 (33.0)
GPT-6.1-Sol	85.59 (17.26)	66.02 (21.02)	73.30 (16.44)	31.2 (10.6)	135.2 (46.3)	103.4 (30.4)
GLM 5.3	75.08 (21.22)	63.18 (21.56)	67.69 (17.67)	89.9 (28.0)	238.7 (96.6)	116.3 (49.4)
HY4-Preview	77.27 (20.63)	60.00 (21.38)	66.55 (17.72)	70.7 (21.3)	177.9 (49.2)	86.0 (23.3)
Kimi-K3	72.64 (22.83)	62.89 (23.43)	66.39 (19.14)	36.4 (14.4)	97.5 (35.1)	48.0 (16.8)
DeepSeek-V4.1-Flash	72.30 (19.72)	57.34 (20.29)	62.95 (16.93)	58.1 (20.2)	150.7 (48.0)	74.0 (24.3)
Qwen-3.8-27B	66.93 (25.33)	54.93 (22.46)	59.64 (20.16)	55.1 (18.5)	170.4 (39.7)	82.1 (20.7)
Gemini-3.8-Flash	59.00 (21.70)	54.13 (21.18)	56.17 (17.01)	62.3 (15.6)	98.8 (20.9)	50.8 (10.3)
MiniMax-M3	53.39 (20.44)	47.39 (19.85)	49.75 (16.89)	93.8 (32.1)	158.4 (47.2)	62.6 (19.4)

DataWorld-Bench

World-disjoint evaluation

Human preference-based curation selects realistic questions from audited, held-out worlds. Each world contributes a cause and a consequence task.

100tasks

50held-out worlds

16domains

Multi-source filesCause & consequenceObserved data onlyAtomic rubric scoring

Two complementary capabilities

Data computation

Recover analyzable data from heterogeneous files: parsing, cleaning, joins, granularity, and metric definitions.

Insight discovery

Answer the question with supported comparisons, relationships, magnitudes, heterogeneity, and limitations.

Rubrics ground selected targets, not every valid insight. Unlisted findings are not penalized merely for being unlisted.

Task-solving evaluation uses mini-swe-agent v2.4.6. Generation and task synthesis use Claude Code. Test worlds are excluded from training.

04 / Training utility

Synthetic worlds.
Transfer beyond them.

RL-zero training produces DataWorld-35B-A3B from Qwen3.6-35B-A3B using its own rollouts and rubric rewards—without task-solving demonstrations or stronger-model trajectory distillation.

+7.06DataWorld-Bench · points

+14.28DAComp · points

+20.90DABstep-Hard · points

+12.13JobBench · points

Manuscript-reported gains under matched evaluation. Benchmarks retain their own scoring rules; gains should be interpreted within each benchmark, not compared across benchmarks. These results are not reproduced by the repository quick start.

Table 3 / Final capability

Same backbone. Four benchmarks.

Compare DataWorld-35B-A3B with its base and three same-base models. Each panel uses a zero-to-100 score axis; interpret scores within its own benchmark.

Matched evaluation

Gray: base · Blue: comparison models · Green: DataWorld.

Hover, focus, or tap a row to see its full model name, mean and SD.

DataWorld has the highest reported mean among these same-base models on all four benchmarks. The comparisons assess overall training utility—not an isolated effect of the dataset or optimization recipe. Higher scores also accompany more model calls and response tokens than the base; see the cost columns below.

View Table 3 · all scores, SDs, rounds & response tokens

Mean (SD); gains in score points relative to the base. Gains are transcribed as reported and may differ slightly from subtracting displayed, rounded means. Response tokens are in thousands. Comparison charts display score means, not confidence intervals.
Model	JobBench			DAComp			DABstep-Hard			DataWorld-Bench
Model	Score ↑	Rounds	Resp. tokens (k)	Score ↑	Rounds	Resp. tokens (k)	Score ↑	Rounds	Resp. tokens (k)	Score ↑	Rounds	Resp. tokens (k)
Qwen3.6-35B-A3B	18.31 (18.12)	38.7 (17.4)	60.4 (22.1)	53.72 (20.35)	35.6 (17.7)	41.8 (19.6)	25.66 (43.73)	28.6 (18.0)	15.1 (15.3)	37.64 (11.62)	36.5 (13.9)	38.4 (19.2)
DataMind-35B-A3B	22.93 (22.33) +4.63	36.6 (12.7)	73.9 (28.0)	54.79 (20.49) +1.07	37.6 (14.1)	47.7 (19.0)	31.22 (46.40) +5.56	40.3 (17.6)	26.3 (16.8)	37.36 (11.14) −0.29	34.4 (9.6)	42.0 (17.2)
BigBang-v1	20.84 (20.69) +2.53	68.9 (23.4)	84.3 (30.1)	54.31 (20.93) +0.58	45.1 (22.9)	38.2 (18.4)	34.66 (47.65) +8.99	37.2 (24.5)	35.5 (34.0)	40.05 (12.06) +2.41	58.4 (27.8)	46.7 (18.1)
Occamy-1.0	30.18 (23.79) +11.87	79.2 (41.7)	96.4 (38.9)	64.86 (20.89) +11.13	49.1 (22.5)	45.4 (21.6)	44.44 (49.76) +18.78	31.0 (21.3)	26.3 (28.4)	42.38 (14.06) +4.74	61.5 (25.8)	58.2 (28.3)
DataWorld-35B-A3B	30.43 (23.92) +12.13	63.6 (28.9)	88.1 (29.1)	68.00 (18.90) +14.28	78.4 (25.0)	65.9 (21.7)	46.56 (49.95) +20.90	43.2 (22.3)	36.3 (28.0)	44.70 (13.94) +7.06	102.8 (22.6)	68.4 (17.6)

Original Figure 4: scores on JobBench, DAComp, DABstep-Hard and DataWorld-Bench from the pre-RL base through training checkpoints 40 to 90. Final scores are 0.3043, 0.6800, 0.4656 and 0.4470, respectively. — Click to enlarge. Figure 4 uses unscaled scores (for example, 0.4470 corresponds to 44.70 in Table 3). Each panel has its own vertical range, so apparent slopes and curve heights should not be compared across benchmarks. Percentage labels in the figure are relative gains, distinct from absolute score-point gains.

05 / Explore & build

A foundation for
your next data world.

Explore the synthesis engine on GitHub, find datasets and model releases in our Hugging Face collection, and read the paper on arXiv.

Open-source engine

Code & documentation

DAG generation, world synthesis, task rubrics, and Harbor export. Includes examples and bilingual usage documentation.

Explore on GitHub →

Data release

Benchmark & training data

Curated analytical tasks with environments and grounded rubrics. Not bundled in the current code repository.

Explore on Hugging Face →

Model release

DataWorld-35B-A3B

A data agent trained with RL-zero on synthesized worlds. Model weights are not bundled in the code repository.

Explore on Hugging Face →

Correspondence: Feng Hong · Han-Jia Ye

06 / Citation

Cite Data World.

If you find our work useful, please cite the paper on arXiv using the BibTeX entry below.

@misc{li2026dataworldscalingdata,
      title={Data World: Scaling Data Environments from Scratch},
      author={Zhanli Li and Zeyuan Xu and Chaoxu Pang and Zeyuan Liu and An-Yang Ji and Te Sun and Kongcheng Zhang and Wenbo Xu and Xubin Yue and Shipeng Luo and Junjie Zhao and Yingfan Hua and Jingqing Ruan and Zhongda Su and Feng Hong and Zhizhao Zeng and Ke Zeng and Xunliang Cai and Han-Jia Ye},
      year={2026},
      eprint={2610.21656},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2610.21656},
}