Skip to content
BranchesTags

Latest commit

 

History

38 Commits

Folders and files

NameName
Last commit message
Last commit date
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
README_zh.md
README_zh.md
 
 

Repository files navigation

Awesome Instruction Datasets

Awesome Last Updated

中文 | English

Contents

Introduction

"Welcome to 'awesome-prompt-datasets', a comprehensive collection of high-quality open-source instruction tuning datasets to train chat-based LLMs (ChatGPT, LLaMA, Alpaca).

Instruction Tuning / Reinforcement Learning from Human Feedback (RLHF) Dataset is a key component of instruction-following LLMs such as ChatGPT. This repo is dedicated to providing a comprehensive list of datasets used for instruction tuning in various LLMs, making it easier for researchers and developers to access and utilize these resources.

With 'awesome-prompt-dataset', you can accelerate your research and development in NLP and unlock new opportunities for innovation. Let's explore the possibilities together!"

What's New (2023–2026)

The instruction-tuning landscape has evolved rapidly. Below are representative high-quality datasets that have emerged since this list was first created, covering general SFT, RLHF/preference, DPO, code/math reasoning, agent/tool use, and Chinese resources. For Chinese datasets, also see README_zh.md for ModelScope links.

Notable additions:

  • SFT: LIMA, UltraChat, OpenOrca/SlimOrca, OpenHermes 2.5, WizardLM/Evol-Instruct, MetaMathQA, Airoboros, Platypus, No Robots, Magpie, Infinity Instruct, Deita, WildChat, Capybara, Tulu 3 SFT Mix, Smol-Smoltalk, PersonaHub, OpenMathInstruct-1, DART-Math, LIMO, Code-Feedback, FineWeb-Edu, OpenCodeInstruct, Nemotron-Cascade-2-SFT-Data, KIMI-K2.5-1000000x, CodeX-7M-Non-Thinking, Dolci-Instruct-SFT, AgentTrove, ToolMind, AM-Thinking-v1-Distilled, SYNTHETIC-2-SFT-verified, OpenThoughts-114k / OpenThoughts3-1.2M, OpenR1-Math-220k, NuminaMath-CoT, Bespoke-Stratos-17k.
  • RLHF / Preference: Anthropic HH-RLHF, Stanford SHP, UltraFeedback, HelpSteer/HelpSteer2/HelpSteer2-Preference, NVIDIA HelpSteer3-Preference, Nemotron-Cascade-2-RL-Data, TaskTrove, SYNTHETIC-2-RL, UltraInteract_preference, RLSTACK.
  • DPO: Orca-DPO-Pairs, UltraFeedback-Binarized, DPO-Mix-7K, Llama3-UltraFeedback-ArmoRM, ChatML-DPO-Pairs, Magpie-Air-DPO, Magpie-Pro-DPO, Tulu 3 Preference, RLSTACK, UltraInteract_preference, Ling-Coder-DPO, DAPO-Math-17k.
  • Code / Math: MetaMathQA, OpenMathInstruct-1, DART-Math, LIMO, Code-Feedback, OpenCodeInstruct, CodeX-7M-Non-Thinking, OpenR1-Math-220k, NuminaMath-CoT, Bespoke-Stratos-17k, DAPO-Math-17k.
  • Agent / Tool Use: AgentTrove, ToolMind, Nemotron-Cascade-2-SFT-Data (agentic subset).
  • Chinese / ModelScope: COIG-CQIA, MSAgent, DeepCtrl-SFT, Infinity Instruct, Alpaca-GPT4 Chinese, HC3-Chinese, Magpie-Qwen2-Pro-200K-Chinese, Chinese-DeepSeek-R1-Distill-110k-SFT, Seq-Monkey, smoltalk-chinese, OpenThoughts3-1.2M.

The list is now organized into SFT datasets, RLHF datasets, and DPO datasets sections.

Quick Index by Category

Category Where to find Highlights
General SFT SFT Statistics • General SFT Details Alpaca, LIMA, UltraChat, OpenOrca, OpenHermes 2.5, WizardLM, Magpie, No Robots, Deita, WildChat, Capybara, Tulu 3, Smol-Smoltalk, PersonaHub, Nemotron-Cascade-2, Dolci-Instruct-SFT, KIMI-K2.5-1000000x
Code & Math SFT Statistics • Code & Math Details Code Alpaca, MetaMathQA, OpenMathInstruct-1, DART-Math, LIMO, Code-Feedback, OpenCodeInstruct, CodeX-7M-Non-Thinking, OpenR1-Math-220k, NuminaMath-CoT, Bespoke-Stratos-17k
Multilingual & Chinese SFT Statistics • Multilingual & Chinese Details Chinese-LLaMA-Alpaca, BELLE, COIG, COIG-CQIA, MSAgent, DeepCtrl-SFT, Infinity Instruct, Chinese-DeepSeek-R1-Distill, Seq-Monkey
Agent & Tool Use SFT Statistics • Agent & Tool Use Details AgentTrove, ToolMind, MSAgent
RLHF / Preference RLHF Statistics • RLHF Details HH-RLHF, SHP, UltraFeedback, HelpSteer family, Nemotron-Cascade-2-RL, TaskTrove, SYNTHETIC-2-RL
DPO DPO Statistics • DPO Details Orca-DPO-Pairs, UltraFeedback-Binarized, DPO-Mix-7K, Llama3-UltraFeedback-ArmoRM, Magpie DPO, RLSTACK, UltraInteract_preference, Ling-Coder-DPO
Chinese ModelScope README_zh.md COIG-CQIA, MSAgent, DeepCtrl-SFT, Infinity Instruct, Chinese-DeepSeek-R1-Distill, Seq-Monkey, smoltalk-chinese, OpenThoughts3-1.2M

How to use this list

Each dataset is tagged by Language (EN = English, CN = Chinese, ML = multilingual), Task (MT = multi-task, TS = task-specific), and Generation (HG = human-generated, SI = self-instruct, MIX = mixed, COL = collection).

  • Use the Table of Contents or Quick Index by Category to jump to a section.
  • Use Ctrl+F / Cmd+F to search for a dataset name, model, or paper.
  • Tables link directly to the dataset card; detail sections provide size, license, source model, and paper links.

SFT Datasets

Referring to this (@yaodongC), we labeled each collected dataset according to the following rules:

(Lang) Lingual-Tags:

  • EN: Instruction datasets in English
  • CN: Instruction datasets in Chinese
  • ML: [Multi-lingual] Instruction datasets in multiple languages

(Task) Task-Tags:

  • MT: [Multi-task] Datasets containing multiple tasks
  • TS: [Task-specific] Datasets tailored for specific tasks

(Gen) Generation-method:

  • HG: [Human Generated Dataset] Datasets created by humans
  • SI: [Self-Instruct] Datasets generated using self-instruct methods
  • MIX: [Mixed Dataset] Dataset contains both human and machine generated data
  • COL: [Collection of Dataset] Dataset made from a collection of other datasets

SFT Statistics

General SFT

Project Datasets Org Nums Lang Task Gen Type Src Url
Chain of Thought cot_data |few_shot_data Google 74771 EN/CN MT HG instruct with cot reasoning annotating CoT on existing data download
GPT4all nomic-ai/gpt4all-j-prompt-generations nomic-ai 806199 EN MT COL code, storys and dialogs distillation from GPT-3.5-turbo download
GPTeacher GPT-4 General-Instruct |Roleplay-Instruct |Code-Instruct | Toolformer teknium1 29013 EN MT SI general, roleplay, toolformer GPT-4 & toolformer download
Guanaco JosephusCheung/GuanacoDataset JosephusCheung 534610 ML MT SI various linguistic tasks text-davinci-003 download
alpaca tatsu-lab/alpaca tatsu-lab 52002 EN MT SI general instruct text-davinci-003 download
AlpacaDataCleaned yahma/alpaca-cleaned yahma 52k EN MT SI general instruct text-davinci-003 download
Natural Instructions Allen AI 61 task|1.5k task Allen AI 5040134 ML MT COL diverse nlp tasks human annotated datasets collection download
prosocial dialog allenai/prosocial-dialog allenai 165681 EN TS MIX dialogue GPT-3 rewrites questions + humans feedback manually download
finance_en gbharti/finance-alpaca 68912 EN TS COL financial related qa GPT3.5 download
instruct swype/instruct 888969 EN MT COL augmented of GPT4All, Alpaca, open-source Meta datasets augmentation performed using the advanced NLP tools provided by AllenAI download
webGPT openai/webgpt_comparisons openai 18994 EN TS MIX information retrieval (IR) QA fine-tuned GPT-3, each instruction has two outputs, select better one download
dolly 2.0 databricks/databricks-dolly-15k databricks 15015 EN TS HG closed QA , summarization and etc, Wikipedia as references human annotated download
mosaicml/llm-foundry mosaicml/dolly_hhrlhf mosaicml 59.3K EN TS HG This dataset is a combination of Databrick's dolly-15k dataset and a filtered subset of Anthropic's HH-RLHF. human annotated
baize 白泽 alpaca_chat_data.json |medical_chat_data.json | quora_chat_data.json |stackoverflow_chat_data.json project-baize 653699 EN MT COL a collection from Alpaca, Quora, StackOverFlow and MedQuAD questions human annotated datasets collection download
hh-rlhf Anthropic/hh-rlhf Anthropic 284517 EN TS MIX dialogue dialog between human and RLHF models download
OIG(part) laion/OIG laion 49237 EN MT COL created from various tasks, such as question and answering using data augmentation, human annotated datasets collection download
camel | 骆驼 camel-ai/code|camel-ai/biology |camel-ai/physics |camel-ai/chemistry |camel-ai/math camel-ai 760620 EN MT SI Role-Playing conversations in AI Society, Code, Math, Physics, Chemistry, Biolog gpt-3.5-turbo download
FLAN-Muffin Muennighoff/flan 1764800 EN MT COL 60 nlp tasks human annotated datasets collection download
ShareChat RyokoAI/ShareGPT52K RyokoAI 1663241 EN MT MIX general instruct crowdsourcing to collect conversations between people and ChatGPT (ShareGPT) download
Auto CoT kojima-takeshi188/zero_shot_cot/dataset |kojima-takeshi188/zero_shot_cot/log amazon-science EN download
MOSS（复旦 Moss） fnlp/moss-002-sft-data| moss-003-sft-data fnlp 1583595 EN/CN SI download
ultrachat stingning/ultrachat thnlp 28247446 EN download
StackLLaMA lvwerra/stack-exchange-paired todo EN HG
Self-Instruct yizhongw/self-instruct 82 K EN SI SI
Zhihu-KOL Zhihu-KOL Openassisent 100 w SI HG Zhihu data for training Open Assitant
stanfordnlp/SHP stanfordnlp/SHP stanfordnlp 385 k EN MT HG human preferences over responses
LAION-AI/Open-Assistant OpenAssistant/oasst1 Openassisent 84.4k EN MT HG OpenAssistant Conversations Dataset (OASST1) human-generated, human-annotated
akoksal/LongForm akoksal/LongForm akoksal/LongForm 30k EN SI HG 们从现有语料库（如 C4 和维基百科）中选择一组不同的人工文档，并通过 LLM 为给定的文档生成指令。
sail-sg/symbolic-instruction-tuning sail/symbolic-instruction-tuning sail-sg 800K ML SI Human Synthetic Examples
医疗问答 michael-wzhu/PromptCBLUE michaelwzhu/ChatMed_Consult_Dataset michael-wzhu 110113 CN SI 互联网上的医疗问诊问题(110,113)，反映了真实世界的不同用户/患者的医疗问诊需求。目前response都是由OpenAI GPT-3.5引擎回答的。
mbzuai-nlp/LaMini-LM MBZUAI/LaMini-instruction MBZUAI/LaMini-instruction 2.58M EN MT SI 通过离线蒸馏从大型语言模型中提取知识
pCLUE pCLUE 120 万
WizardLM victor123/evol_instruct_70k |evol_instruct_V2_196k WizardLM 70k-196k EN MT SI general instruct evolved for complexity GPT-4 / ChatGPT evolution download
LIMA GAIR/lima Meta AI 1k EN MT HG high-quality human-written instructions Stack Exchange, wikiHow, Reddit, human authors download
UltraChat stingning/ultrachat |HuggingFaceH4/ultrachat_200k THUNLP / HuggingFaceH4 1.5M / 200k EN MT SI multi-turn dialogue GPT-3.5-turbo generated, filtered download
OpenOrca Open-Orca/OpenOrca |Open-Orca/SlimOrca Open-Orca 4.2M / 550k EN MT SI reasoning traces over FLAN-style prompts GPT-4 / GPT-3.5 augmented FLAN Collection download
OpenHermes 2.5 teknium/OpenHermes-2.5 Nous Research / Teknium ~1M EN MT COL curated mixture: SlimOrca, Evol-Instruct, Platypus, ShareGPT, etc. collection of open-source datasets download
Airoboros jondurbin/airoboros-3.2 jondurbin 58.7k EN MT SI customizable self-instruct, uncensored GPT-4 self-instruct download
Platypus garage-bAInd/Open-Platypus garage-bAInd 25k EN TS COL STEM focused (math/science) merging selected datasets download
No Robots HuggingFaceH4/no_robots HuggingFaceH4 10k EN MT HG human-curated across 10 categories skilled human annotators download
Magpie Magpie-Align/Magpie-Pro-1M-v0.1 |Magpie-Qwen2-Pro-200K-Chinese Magpie-Align 1M / 200k EN/CN MT SI alignment data synthesized from aligned LLMs Llama-3 / Qwen2-Instruct auto-generated download
Deita hkust-nlp/deita-10k-v0 HKUST-NLP 10k EN MT COL data-efficient high-quality alignment SFT complexity + quality + diversity selection on open data download
WildChat allenai/WildChat-1M |lmsys/lmsys-chat-1m Allen AI / LMSYS 1M-4.8M ML MT HG real-world human-ChatGPT / multi-model conversations real user interactions on Chatbot Arena & ChatGPT download
Capybara LDJnr/Capybara LDJnr / Nous Research 15k EN MT SI multi-turn reasoning conversations Amplify-Instruct from seed datasets download
Tulu 3 SFT Mix allenai/tulu-3-sft-mixture Ai2 ~939k EN MT COL open post-training SFT mixture curated open-source instruction datasets download
Smol-Smoltalk HuggingFaceTB/smoltalk |HuggingFaceTB/smoltalk2 Hugging Face TB 460k-3.4M EN/ML MT COL instruction data for SmolLM family collection + synthetic data download
PersonaHub proj-persona/PersonaHub Tencent AI Lab 200k personas / 100k+ instruct EN/CN MT SI persona-driven synthetic instruction data 1B persona scaling for diverse synthesis download
FineWeb-Edu HuggingFaceFW/fineweb-edu HuggingFaceFW 1.3T+ tokens ML MT COL high-quality educational web corpus for pretrain/SFT FineWeb filtered for educational content download
Nemotron-Cascade-2-SFT-Data nvidia/Nemotron-Cascade-2-SFT-Data NVIDIA 15.87M EN/ML MT COL large-scale multi-domain SFT (chat, math, science, code, agent, SWE, safety) on-policy distillation from DeepSeek-V3.2, GPT-OSS-120B, Qwen3, Nemotron-Cascade-1 download
KIMI-K2.5-1000000x ianncity/KIMI-K2.5-1000000x ianncity 693K (~5B tokens) EN/ML MT SI reasoning traces distilled from Kimi K2.5 high-reasoning mode Kimi K2.5 synthetic download
Dolci-Instruct-SFT allenai/Dolci-Instruct-SFT Ai2 2.15M ML MT COL OLMo 3 SFT mixture (math, code, multilingual, safety, logic puzzles) curated open data + Ai2 synthetic prompts download
AM-Thinking-v1-Distilled a-m-team/AM-Thinking-v1-Distilled a-m-team (Beike) 1.89M CN/EN MT SI verified reasoning distillation (math, code, chat, science, IF) AM-Thinking-v1 / Qwen3 / DeepSeek-R1 + verifiers download
SYNTHETIC-2-SFT-verified PrimeIntellect/SYNTHETIC-2-SFT-verified PrimeIntellect 105K EN MT SI verified reasoning traces (math, code, formatting, puzzles) DeepSeek-R1-0528 + distributed verification download
OpenThoughts-114k open-thoughts/OpenThoughts-114k OpenThoughts 114K EN MT SI synthetic reasoning (math, science, code, puzzles) DeepSeek-R1 verified download
OpenThoughts3-1.2M open-thoughts/OpenThoughts3-1.2M OpenThoughts 1.2M EN TS SI scaled reasoning dataset (math / code / science) QwQ-32B 16× annotations download

Code & Math

Project Datasets Org Nums Lang Task Gen Type Src Url
Code Alpaca sahil280114/codealpaca 20022 EN TS SI code generation, editing, optimization text-davinci-003 download
MetaMathQA meta-math/MetaMathQA MetaMath 395k EN TS SI mathematical reasoning bootstrapped from GSM8K & MATH via GPT-3.5 download
OpenCodeInstruct OpenCoder-llm/OpenCoder-llm OpenCoder / NVIDIA 5M EN TS SI large-scale code instruction tuning synthetic code data with execution verification download
OpenMathInstruct-1 nvidia/OpenMathInstruct-1 NVIDIA 1.8M EN TS SI mathematical problem-solving GSM8K & MATH synthetic solutions download
DART-Math hkust-nlp/DART-Math HKUST-NLP 600k+ EN TS SI difficulty-aware math instruction tuning rejection tuning from MATH/GSM8K with difficulty control download
LIMO GAIR/LIMO GAIR-NLP 817 EN TS COL high-quality reasoning elicitation curated MATH/AIME/NuminaMath with expert CoT download
Code-Feedback m-a-p/Code-Feedback |m-a-p/CodeFeedback-Filtered-Instruction M-A-P 200k+ EN TS COL code instruction with execution feedback collection of code datasets with test feedback download
CodeX-7M-Non-Thinking Modotte/CodeX-7M-Non-Thinking Modotte 7.36M EN TS SI large curated code instruction pairs without reasoning chains Modotte curation + synthetic generation download
OpenR1-Math-220k open-r1/OpenR1-Math-220k Hugging Face 220K EN TS SI math reasoning traces over NuminaMath 1.5 DeepSeek R1 + Math-Verify / Llama judge download
NuminaMath-CoT AI-MO/NuminaMath-CoT AI-MO 860K EN/CN TS COL math problems with CoT solutions (K-12 to olympiad) OCR + translation + realignment download
Bespoke-Stratos-17k bespokelabs/Bespoke-Stratos-17k Bespoke Labs 17K EN TS SI high-quality reasoning distillation (math / code / science) DeepSeek-R1 rejection sampling download

Multilingual & Chinese

Project Datasets Org Nums Lang Task Gen Type Src Url
HC3 Hello-SimpleAI/HC3 Hello-SimpleAI | 万得资讯 37175 EN/CN TS MIX dialogue evaluation human or ChatGPT download
HC3-Chinese Hello-SimpleAI/HC3-Chinese Hello-SimpleAI|万得资讯 13k CN TS MIX dialogue evaluation human or ChatGPT
Chinese-LLaMA-Alpaca alpaca_data_zh_51k ymcui(讯飞) 51k CN MT SI general instruct text-davinci-003
Luotuo-Chinese-LLM 骆驼 trans_chinese_alpaca_data LC1332(商汤) 52k CN MT SI general instruct text-davinci-003
belle_cn BelleGroup/train_1M_CN |BelleGroup/train_0.5M_CN BelleGroup(链家) 1079517 CN TS/MT SI general, mathematical reasoning, dialogue text-davinci-003 download
instinwild instinwild_ch | instinwild_en 52191 EN/CN MT SI generation, open-qa, mind-storm text-davinci-003 download
华驼(HuaTuo) 中文医学知识 |肝癌 SCIR-HI(哈工大) 8K CN TS SI 公开和自建的中文医学知识库 GPT3.5
xP3 bigscience/xP3 bigscience 78883588 ML MT COL a collection of prompts & datasets across 46 of languages & 16 NLP tasks human annotated datasets collection download
firefly YeungNLP/firefly-train-1.1M 1649398 CN MT COL 23 nlp tasks human annotated datasets collection download
Alpaca_GPT4 alpaca_gpt4_data|alpaca_gpt4_data_zh |comparison_data_v2 微软 52002 EN/CN MT SI general instruct generated by GPT-4 using Alpaca download
GAOKAO Fill-in-the-blank_Questions | Multiple-choice_Questions | Open-ended_Questions OpenLMLab 2785 CN MT COL Multiple-choice, Fill-in-the-blank and Open-ended questions from examination human annotated download
COIG COIG BAAI|智源 298428 CN MT COL collect fron Exam, Translated, Human Value Alignment Instructions and Counterfactural Correction Multi-round Chat using automatic tool and manual verification download
Infinity Instruct AI-ModelScope/Infinity-Instruct BAAI / FlagOpen 7M+ EN/CN MT COL large-scale general instruction collection collection + synthesis download
COIG-CQIA m-a-p/COIG-CQIA |modelscope/COIG-CQIA 01.AI / M-A-P 48k CN MT COL high-quality Chinese Q&A and articles Chinese internet data, cleaned + manually reviewed download
DeepCtrl-SFT deepctrl/deepctrl-sft-data DeepCtrl 12M CN/EN MT MIX safe general-domain SFT data curated + safety-filtered Chinese/English data download
Chinese-DeepSeek-R1-Distill-110k-SFT Congliu/Chinese-DeepSeek-R1-Distill-data-110k |liucong/Chinese-DeepSeek-R1-Distill-data-110k Cong Liu 110k CN MT SI Chinese reasoning SFT distilled from DeepSeek-R1-671B math/exam/STEM/general Chinese reasoning download
Seq-Monkey ddzhu123/seq-monkey Mobvoi / 出门问问 10B tokens CN MT COL Chinese pre-training corpus (general/web/QA/code) Chinese web, encyclopedia, Q&A, blogs, books, code download

Agent & Tool Use

Project Datasets Org Nums Lang Task Gen Type Src Url
GPT4Tools gpt4tools_71k.json StevenGrove 71446 EN MT SI a collection of tool-related instructions gpt-3.5-turbo download
MSAgent / MSAgent-Bench iic/MSAgent-Bench |modelscope/MSAgent-Bench Alibaba / ModelScope 600k+ CN/EN MT COL tool-use and agent capabilities API calling, multi-turn tool interactions download
AgentTrove open-thoughts/AgentTrove OpenThoughts 1.7M EN MT COL agentic interaction traces (code repair, shell, math, CP, computer-use) 219 source datasets via terminus-2 / Harbor download
ToolMind Nanbeige/ToolMind Nanbeige Lab 369K EN TS MIX reasoning-enhanced function-calling / tool-use trajectories multi-agent simulation + curated open data download

SFT Dataset Details

General SFT

Alpaca -Stanford

The Alpaca of the Stanford release is a fine-tuning model for instruct-tuning based on the Meta Ai LLaMA model.

Alpaca automatically generated 52k instruction data using GPT-3.5 and used it to fine-tune the LLaMA model. Experimental results show that it can reach or even exceed the performance of GPT-3.5 on some tasks.

Instruction in the Wild

Instruction Tuning is a key component of ChatGPT. OpenAI used their user-based Instruction dataset, but unfortunately, this dataset is not open-sourced. Self-Instruct released a small instruction dataset including 175 instructions written by human labors. Standford Alpaca Team generated 52K instructions by text-davinci-003 model based on the the 175 seed instructions above.

This project targets on a larger and more diverse instruction dataset. To this end, we collected 429 instructions from ChatGPT usage screenshots and released both English and Chinese versions. We found these instructions are very diverse even if the scale is still small. We follow Alpaca to generate 52K instructions and their responses. All data can be found in data dir.

Note: This is an ongoing project. We are still collecting and improving our data. We release this dataset as early as possible to speedup our LLM research. We will also release a whitepaper soon.

JosephusCheung/GuanacoDataset

  • Data generation model: text-davinci-003
  • Cost: $6000

52K instruction data generated from modified self-instruct pipeline with human written 429 seed task.

Stanford Human Preferences Dataset (SHP)

SHP is a dataset of 385K collective human preferences over responses to questions/instructions in 18 different subject areas, from cooking to legal advice. The preferences are meant to reflect the helpfulness of one response over another, and are intended to be used for training RLHF reward models and NLG evaluation models (e.g., SteamSHP).

Each example is a Reddit post with a question/instruction and a pair of top-level comments for that post, where one comment is more preferred by Reddit users (collectively). SHP exploits the fact that if comment A was written after comment B but has a higher score nonetheless, then A is ostensibly more preferred to B. If A had been written before B, then we could not conclude this, since its higher score could have been the result of more visibility. We chose data where the preference label is intended to reflect which response is more helpful rather than which is less harmful, the latter being the focus of much past work.

How is SHP different from Anthropic's HH-RLHF dataset? Most notably, all the data in SHP is naturally occurring and human-written, whereas the responses in HH-RLHF are machine-written, giving us two very different distributions that can complement each other.

Hello-SimpleAI/HC3

Hello-SimpleAI/HC3-Chinese

allenai/prosocial-dialog

allenai/natural-instructions

PhoebusSi/Alpaca-CoT

  • Summary: A datset for Chain-of-Thoughts reasoning based on LLaMA and Alpaca. Note: Their repository will continuously collect various instruction tuning datasets. Github Repo
  • paper: N/A
  • Cost: N/A

nomic-ai/gpt4all

bigscience/xP3

teknium1/GPTeacher

  • Summary: A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer
  • Data generation model: GPT-4
  • paper: N/A
  • Cost: N/A

thunlp/UltraChat

  • Summary: UltraChat aims to construct an open-source, large-scale, and multi-round dialogue data. The first part of UltraChat (i.e., the Questions about the World sector) is released, which contains 280k diverse and informative dialogues. More dialogues about writing and creation, assistance on existing materials are to come.
  • Data generation model: GPT-3.5-turbo
  • paper: N/A
  • Cost: N/A

cascip/ChatAlpaca

  • Summary: Based on the Stanford Alpaca data, ChatAlpaca extends the data to multi-turn instructions and their corresponding responses. More data (20k) and the Chinese translated version are to come.
  • Data generation model: GPT-3.5-turbo
  • paper: N/A
  • Cost: N/A
  • Related: (tatsu-lab/Alpaca)|52K|EN|MT|SI

YeungNLP/firefly-train-1.1M)

  • Summary: Chinese datasets of 23 tasks combined with human-written instruction templates.
  • Data generation model: N/A
  • paper: N/A
  • Cost: N/A

orhonovich/unnatural-instructions

Instruction-Tuning-with-GPT-4/GPT-4-LLM

databrickslabs/dolly

  • Summary: This datset was generated by thousands of Databricks employees in several of the behavioral categories outlined in the InstructGPT paper, including brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization.
  • Data generation model: N/A
  • paper: Free Dolly
  • Cost: N/A

OpenAssistant/oasst1

  • Summary: OpenAssistant Conversations (OASST1), a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages distributed across 66,497 conversation trees, in 35 different languages, annotated with 461,292 quality ratings.
  • Data generation model: N/A
  • paper: OpenAssistant Conversations - Democratizing Large Language Model Alignment
  • Cost: N/A

BELLE/data/1.5M

alpaca_chinese_dataset

Med-ChatGLM/data

  • 下载地址: https://github.com/SCIR-HI/Med-ChatGLM
  • 数据量: 7k
  • 生成方式: 利用GPT3.5接口围绕医学知识库构建问答数据，并设置了多种Prompt形式来充分利用知识
  • 涉及任务: 医学领域相关的问答，包含并发症，高危因素，组织学检查，临床症状，药物治疗，辅助治疗

pCLUE

COIG

https://github.com/FreedomIntelligence/InstructionZoo

https://github.com/lightaime/camel

LIMA

  • Dataset Link
  • Summary: 1,000 carefully curated human-written instructions showing that "less is more for alignment". LIMA demonstrates that short fine-tuning on small, high-quality data can achieve competitive results with much larger synthetic datasets.
  • Data generation model: human generated
  • paper: LIMA: Less Is More for Alignment
  • Cost: N/A

UltraChat / UltraChat 200k

OpenOrca / SlimOrca

OpenHermes 2.5

  • Dataset Link
  • Summary: A ~1M sample curated mixture of high-quality open-source instruction datasets including SlimOrca, Evol-Instruct, Platypus, ShareGPT, Airoboros, GPTeacher and others. Widely used for training generalist chat models.
  • Data generation model: COL
  • paper: N/A
  • Cost: N/A

WizardLM / Evol-Instruct

Airoboros

  • Dataset Link
  • Summary: Customizable self-instruct pipeline producing high-quality instruction data across general, code, roleplay and other domains. Version 3.2 contains ~58.7k samples.
  • Data generation model: GPT-4
  • paper: N/A
  • Cost: N/A

No Robots

  • Dataset Link
  • Summary: 10k high-quality, fully human-curated instruction dataset modeled after OpenAI's InstructGPT paper. Covers 10 categories including generation, QA, coding, summarization and more.
  • Data generation model: human generated
  • paper: N/A
  • Cost: N/A

Magpie

Deita

  • Dataset Link
  • Summary: A 10K-sample data-efficient instruction tuning dataset selected by complexity, quality, and diversity scorers. Deita shows that principled data selection can match or exceed models trained on 10x more data.
  • Data generation model: COL
  • paper: What Makes Good Data for Alignment?
  • Cost: N/A

WildChat

Capybara

  • Dataset Link
  • Summary: ~15K multi-turn reasoning conversations generated using Amplify-Instruct, expanding high-quality seed instructions into deep, multi-turn dialogues across science, math, logic, and culture.
  • Data generation model: GPT-4
  • paper: N/A
  • Cost: N/A

Tulu 3 SFT Mix

Smol-Smoltalk

  • Dataset Link
  • Dataset Link 2
  • Summary: Instruction dataset for the SmolLM family. smoltalk2 (~3.4M samples) adds OpenThoughts, Tulu 3, and multilingual data for training small but capable chat models.
  • Data generation model: COL
  • paper: N/A
  • Cost: N/A

PersonaHub

Nemotron-Cascade-2-SFT-Data

  • Dataset Link
  • Paper
  • Summary: ~15.87M-sample multi-domain SFT dataset used to train Nemotron-Cascade-2-30B-A3B. Covers chat (~9.3M), math (~2.9M), science (~1.8M), instruction following (~820K), conversational agent (~548K), terminal agent (~324K), SWE (~102K), and safety. Responses were generated via on-policy distillation from DeepSeek-V3.2, GPT-OSS-120B, Qwen3-235B-A22B, and Nemotron-Cascade-1.
  • Data generation model: DeepSeek-V3.2, GPT-OSS-120B, Qwen3-235B-A22B, Nemotron-Cascade-1
  • License: NVIDIA Open Model License

KIMI-K2.5-1000000x

  • Dataset Link
  • Summary: ~693K reasoning traces (~5B tokens) distilled from Moonshot AI's Kimi K2.5 in high-reasoning mode. Composition is roughly 50% code, 20% science, 15% math, plus CS, logic, creative writing, and multilingual STEM subsets.
  • Data generation model: Kimi K2.5
  • License: Apache 2.0

Dolci-Instruct-SFT

  • Dataset Link
  • Paper
  • Summary: 2.15M instruction samples used to train OLMo 3 7B Instruct SFT. Mixes existing sources (OpenThoughts 3, FLAN v2, Aya, Tulu 3 personas, WildChat upgraded with GPT-4.1, SciRIFF, etc.) with new Ai2 prompts for precise instruction following, Python algorithms, logic puzzles, verifiable reasoning, and tool use.
  • Data generation model: COL
  • License: ODC-BY

AM-Thinking-v1-Distilled

  • Dataset Link
  • Paper
  • Summary: 1.89M verified reasoning examples distilled from AM-Thinking-v1 (and also Qwen3-235B-A22B / DeepSeek-R1). Covers math (~29.5%), code (~17.1%), general chat (~41.8%), and science/IF/dialogue. Each response is verified with Math-Verify, sandbox execution, and model judges.
  • Data generation model: AM-Thinking-v1, Qwen3-235B-A22B, DeepSeek-R1
  • License: research-only

SYNTHETIC-2-SFT-verified

  • Dataset Link
  • Blog
  • Summary: 105K verified reasoning traces from the SYNTHETIC-2 collection. Responses are generated by DeepSeek-R1-0528 and filtered to keep only correct solutions (reward = 1 or >0.7). Tasks include code output prediction, Pydantic/JSON formatting, sentence unscrambling, ASCII trees, and other verifiable reasoning problems.
  • Data generation model: DeepSeek-R1-0528

OpenThoughts-114k

  • Dataset Link
  • Paper
  • Summary: 114K high-quality synthetic reasoning examples across math, science, code, and puzzles. Used to train OpenThinker-7B / 32B. Solutions are generated by DeepSeek-R1 and verified.
  • Data generation model: DeepSeek-R1
  • License: Apache 2.0

OpenThoughts3-1.2M

  • Dataset Link
  • Paper
  • Summary: Third iteration of the OpenThoughts series with 1.2M examples (850K math, 250K code, 100K science). Each question is annotated 16× with QwQ-32B and rigorously filtered. Used to train OpenThinker3-7B.
  • Data generation model: QwQ-32B
  • License: Apache 2.0

Code & Math

MetaMathQA

OpenMathInstruct-1

DART-Math

LIMO

  • Dataset Link
  • Summary: Only 817 curated high-quality reasoning examples that elicit strong complex reasoning (AIME 57%, MATH 95%) from a 32B base model, demonstrating "less is more for reasoning".
  • Data generation model: COL
  • paper: LIMO: Less is More for Reasoning
  • Cost: N/A

Code-Feedback

OpenCodeInstruct

CodeX-7M-Non-Thinking

  • Dataset Link
  • Summary: 7.36M curated code instruction pairs emphasizing direct code solutions (no step-by-step reasoning chains). Covers Python, Java, C++, JavaScript, algorithms, web dev, ML/AI, databases, and competitive programming.
  • Data generation model: COL / Modotte curation
  • License: Apache 2.0

OpenR1-Math-220k

  • Dataset Link
  • GitHub
  • Summary: 220K math problems from NuminaMath 1.5 with 2–4 reasoning traces per problem generated by DeepSeek R1. Traces are verified with Math-Verify (~88%) and Llama-3.3-70B-Instruct as judge (~12%). Includes default (~94K) and extended (~131K) splits.
  • Data generation model: DeepSeek-R1
  • License: Apache 2.0

NuminaMath-CoT

  • Dataset Link
  • Summary: ~860K math problems with Chain-of-Thought solutions, spanning Chinese K-12 exercises to US/international olympiad problems. Sources include cn_k12, olympiads, AoPS, GSM8K, MATH, and synthetic data.
  • Data generation model: COL
  • License: Apache 2.0

Bespoke-Stratos-17k

  • Dataset Link
  • Blog
  • Summary: 17K high-quality reasoning-distillation examples (5K code from APPs/TACO, 10K math from AIME/MATH/Olympiads, 1K science/puzzles from STILL-2). Uses DeepSeek-R1 as teacher and gpt-4o-mini for filtering, improving correct-solution retention to ~73%.
  • Data generation model: DeepSeek-R1
  • License: Apache 2.0

Multilingual & Chinese

xP3

firefly

  • Dataset Link
  • Summary: Chinese datasets of 23 tasks combined with human-written instruction templates.
  • Data generation model: N/A
  • paper: N/A
  • Cost: N/A

COIG

Infinity Instruct

COIG-CQIA

MSAgent / MSAgent-Bench

  • Dataset Link
  • ModelScope Link
  • Summary: Chinese/English agent tool-use dataset for training LLMs to call APIs and use external tools in multi-turn conversations.
  • Data generation model: MIX
  • paper: N/A
  • Cost: N/A

DeepCtrl-SFT

  • ModelScope Link
  • Summary: 12M safe Chinese/English general-domain SFT entries (~3B tokens). Designed for small-scale Chinese LLM instruction fine-tuning with safety filtering.
  • Data generation model: MIX
  • paper: N/A
  • Cost: N/A

Chinese-DeepSeek-R1-Distill-110k-SFT

  • Dataset Link
  • ModelScope Link
  • Summary: 110K Chinese reasoning SFT samples distilled from the full DeepSeek-R1-671B model. Covers math (~36.5K), exam (~2.4K), STEM (~12K), and general Chinese reasoning (~58K).
  • Data generation model: DeepSeek-R1-671B
  • paper: N/A
  • Cost: N/A

Seq-Monkey

  • ModelScope Link
  • Summary: ~10B token Chinese pre-training corpus released by Mobvoi, including web, encyclopedia, Q&A, blogs, books, and code. Mainly for pre-training/continued pre-training; often combined with instruction data for full Chinese LLM training.
  • Data generation model: COL
  • paper: N/A
  • Cost: N/A

Agent & Tool Use

AgentTrove

  • Dataset Link
  • GitHub
  • Summary: 1.7M agentic interaction traces drawn from 219 source datasets. Covers code repair, shell scripting, mathematical problem solving, competitive programming, and general computer-use tasks. Formatted as ShareGPT-style conversations with original_source, original_teacher, and reward metadata. Companion RL/evaluation dataset: TaskTrove.
  • Data generation model: COL (terminus-2 / Harbor)
  • License: Apache 2.0

ToolMind

  • Dataset Link
  • Paper
  • Summary: ~369K reasoning-enhanced tool-use trajectories. Combines 160K synthetic multi-agent trajectories over 20K+ functions with 200K augmented open-source function-calling data. Includes two-stage quality filtering and explicit <think> reasoning traces.
  • Data generation model: MIX
  • License: Apache 2.0

RLHF / Preference Datasets

RLHF Statistics

Project Links Org Nums Lang Summary
webgpt_comparisons Openai 19,578 English In the WebGPT paper, the authors trained a reward model from human feedback. They used the reward model to train a long form question answering model to align with human preferences. This is the dataset of all comparisons that were marked as suitable for reward modeling by the end of the WebGPT project. There are 19,578 comparisons in total.
SHP stanfordnlp 349 K English SHP is a dataset of 385K collective human preferences over responses to questions/instructions in 18 different subject areas, from cooking to legal advice. The preferences are meant to reflect the helpfulness of one response over another, and are intended to be used for training RLHF reward models and NLG evaluation models (e.g., SteamSHP).
rlhf-reward-datasets yitingxie 76.3 k English
Dahoas/full-hh-rlhf Dahoas 112 k English Anthropic's HH dataset reformatted into prompt, chosen, rejected samples.
Dahoas/synthetic-instruct-gptj-pairwise Dahoas English
Dahoas/rm-static Dahoas 76.3k English Split of hh-static used for training reward models after supervised fine-tuning.
Anthropic/hh-rlhf Anthropic 22k English This RLHF dataset is an iterated 'online' dataset that includes data from 52B language models. It contains 22k helpfulness comparisons and no red-teaming data.
Instruction-Tuning-with-GPT-4/GPT-4-LLM Instruction-Tuning-with-GPT-4 52k English Ranked responses (Note: Data is evaluated by GPT-4 model NOT human) of Alpaca prompts from three models (GPT-4, GPT-3.5 and OPT-IML) by asking GPT-4 to rate the quality. Author believes "GPT-4 is capable of identifying and fixing its own mistakes, and accurately judging the quality of responses"
thu-coai/Safety-Prompts thu-coai/Safety-Prompts thu-coai 100k Chinese 中文安全prompts，用于评测和提升大模型的安全性，将模型的输出与人类的价值观对齐。
Chatgpt-Comparison-Detection project Hello-SimpleAI/HC3 24.3K English Human ChatGPT Comparison Corpus, 60k human answers and 27K ChatGPT answers for around 24K questions.
UltraFeedback openbmb/UltraFeedback OpenBMB (Tsinghua) 64k English Multi-aspect feedback (instruction-following, truthfulness, honesty, helpfulness) for reward/DPO training.
HelpSteer nvidia/HelpSteer NVIDIA 37k English Multi-attribute helpfulness dataset (helpfulness, correctness, coherence, complexity, verbosity).
HelpSteer2 nvidia/HelpSteer2 NVIDIA 10k English High-quality preference dataset for reward models; SOTA on RewardBench at release.
HelpSteer2-Preference nvidia/HelpSteer2-Preference NVIDIA 10k English Preference annotations with human-written justifications for BT vs regression reward modeling.
HelpSteer3-Preference nvidia/HelpSteer3-Preference NVIDIA 40k+ English Diverse open human-annotated preference data across STEM, coding, and multilingual tasks.
Nemotron-Cascade-2-RL-Data nvidia/Nemotron-Cascade-2-RL-Data NVIDIA multi-subset English RL datasets for Nemotron-Cascade-2 (math, code, reasoning, agentic, instruction following).
TaskTrove open-thoughts/TaskTrove OpenThoughts 750K+ English Agentic task specifications in Harbor format for RL and evaluation; companion to AgentTrove.
SYNTHETIC-2-RL PrimeIntellect/SYNTHETIC-2-RL PrimeIntellect 156k English RL subset of SYNTHETIC-2 with difficulty annotations from Qwen3 and DeepSeek-R1-0528.
UltraInteract_preference openbmb/UltraInteract_preference OpenBMB (Tsinghua) ~219k English Preference trees for multi-turn reasoning and tool use (correct/incorrect trajectories and actions).
RLSTACK H-D-T/RLSTACK Hive-Digital-Technologies 868k English DPO-style preference pairs from Stack Exchange dumps (highest- vs lowest-rated answers).

RLHF Dataset Details

Anthropic/hh-rlhf

HuggingFaceH4/stack-exchange-preferences

stanfordnlp/SHP

  • Summary: Each example is a Reddit post with a question/instruction and a pair of top-level comments for that post, where one comment is more preferred by Reddit users (collectively).
  • Data generation model: N/A
  • paper: N/A
  • Cost: N/A

Instruction-Tuning-with-GPT-4/GPT-4-LLM

  • Summary: Ranked responses (Note: Data is evaluated by GPT-4 model NOT human) of Alpaca prompts from three models (GPT-4, GPT-3.5 and OPT-IML) by asking GPT-4 to rate the quality. Author believes "GPT-4 is capable of identifying and fixing its own mistakes, and accurately judging the quality of responses"
  • Data generation model: GPT-4
  • paper: Instruction Tuning with GPT-4
  • Cost: N/A
  • Related: -(tatsu-lab/Alpaca)|52K|EN|MT|SI

Natural Instruction / Super-Natural Instruction

Allen AI is the first organization to try Instruction as a prompt and fine-tune LLMs. In the Natural Instruction paper, you can basically understand the labeling ideas of the instruction.

In its proposed dataset, 61 and different NLP tasks are included.

Super-Natural Instruction is a super-intensive version of Natural Instruction, which contains more than 1,600 different NLP tasks, and there are more than 76 different types of NLP tasks (such as: classification, extraction, sequence labeling).

BigScience/P3

BigScience is jointly organized by Hugging Face and French CNRS, IDRIS, GENCI, etc. It is one of the largest open source LLMs organizations.

BigScience developed the PromptSource project at the end of 2021, and open sourced a series of toolkits to help researchers build prompts based on existing NLP tasks. So far, the PromptSource project contains more than 2000 prompt templates for 270 NLP tasks.

On this basis, BigScience constructed the P3 dataset. You can find P3 data on Hugging Face Hub, and the data size of P3 is between 100M-1B.

xMTF - BigScience

Based on the English prompt, BigScience extends its prompt to multiple non-English languages.

The project contains 13 NLP tasks and is available in 46 different languages. The corresponding prompt contains an indeterminate number of languages.

After fine-tuning on the basis of multilingual, both BLOOM and T0 have realized the ideal multilingual ability.

HH-RLHF - Anthropic

Claud under Anthropic is one of the main competitors of ChatGPT.

Anthropic has open-sourced the RLHF dataset it uses in its own product line.

The original intention of the HH-RLHF project is to train Helpful and Harmless (HH) LLMs. Therefore, in addition to the quality of the project's responses, whether it is harmful information is also reflected in its human feedback.

The paper records how to use the behavior of the RLHF data Align model to human values, and records the construction method and standards of the data set.

Unnatural Instruction

Using LLMs to independently generate instruction data is an active direction in the field of instruction-tuning.

Unnatural Instruction uses GPT3 (text-davinci-002) to generate 64k instruction prompt data. And use the same model to rewrite the 64k prompt, and finally get 240k instruction data.

The paper shows that the prompts generated by LLMs in Instruct-Tuning show good results, even surpassing models such as T0 that are fine-tuned on P3 and other data.

Self-Instruct

Self-Instruct is also the idea of using LLMs to generate prompts for instruction-tuning. However, a more fine-grained generation process is used.

Concepts such as Task pool and Quality filtering were introduced to partially alleviate the noise problem of self-intrauct type data.

UnifiedSKG - HKU

UnifiedSKG has added knowledge grounding in the Text-to-Text framework, that is, in the prompt-output framework, it has added structured data for assistance.

As an example, some NLP tasks rely heavily on structured knowledge bases/databases. The idea of UnifiedSKG is to serialize the required database and embed it into the prompt. As shown below.

UnifiedSKG represents a direction in the field of LLMs that attempts to use structured knowledge to enhance performance.

Google/Flan Collection

In this project, Google merged its own Flan 2021 data with some open source instruction data (P3, super-natural instruction, etc.).

In Flan Collection's paper, Google also summarizes some key points in Flan series model training/reasoning, which may have good reference value.

The Flan Collection compiles datasets from Flan 2021, P3, Super-Natural Instructions, along with dozens more datasets into one place, formats them into a mix of zero-shot, few-shot and chain-of-thought templates

InstructDial

InstructDial is an attempt to fine-tune instructions on a specific task type. Experimental results show that after fine-tuning on dialogue instruction data, the model performs better on dialogue tasks than on very large-scale task sets.

ChatGPT Distillation Data

Public User-Shared Dialogues with ChatGPT (ShareGPT) Around 60K dialogues shared by users on ShareGPT were collected using public APIs. To maintain data quality, we deduplicated on the user-query level and removed any non-English conversations. This leaves approximately 30K examples.

Human ChatGPT Comparison Corpus (HC3) We use both the human and ChatGPT responses from the HC3 english dataset, which contains around 60K human answers and 27K ChatGPT answers for around 24K questions, resulting in a total number of around 87K question-answer examples.

Open Instruction Generalist (OIG).

We use a manually-selected subset of components from the Open Instruction Generalist dataset curated by LAION. Specifically, we use the grade-school-math-instructions, the poetry-to-songs, and the plot-screenplay-books-dialogue datasets. This results in a total of around 30k examples.

OpenAI WebGPT.

In the WebGPT paper, the authors trained a reward model from human feedback. They used the reward model to train a long form question answering model to align with human preferences. This is the dataset of all comparisons that were marked as suitable for reward modeling by the end of the WebGPT project. There are 19,578 comparisons in total.

Each example in the dataset contains a pair of model answers for a question, and the associated metadata. Each answer has a preference score from humans that can be used to determine which of the two answers are better.

OpenAI Summarization.

The OpenAI summarization dataset contains ~93K examples, each example consists of feedback from humans regarding the summarizations generated by a model. Human evaluators chose the superior summary from two options.

UltraFeedback

A large-scale, multi-aspect feedback dataset for training reward models and DPO. It contains ~64k prompts with multiple ranked responses annotated for instruction-following, truthfulness, honesty and helpfulness.

HelpSteer / HelpSteer2 / HelpSteer2-Preference

NVIDIA's family of multi-attribute helpfulness datasets. HelpSteer provides fine-grained ratings across helpfulness, correctness, coherence, complexity and verbosity. HelpSteer2 adds preference pairs for reward model training; HelpSteer2-Preference includes human-written justifications comparing Bradley-Terry and regression reward modeling. HelpSteer3-Preference expands to 40k+ diverse real-world applications including STEM, coding, and multilingual tasks.

Nemotron-Cascade-2-RL-Data

  • Collection Link
  • Paper
  • Summary: Reinforcement-learning datasets used to train Nemotron-Cascade-2-30B-A3B. Includes domain-wise RL data for math, code, reasoning, instruction following, long-context, agentic/SWE, and RLHF-style preference pairs. Released alongside the SFT data and model checkpoints.
  • Data generation model: Nemotron-Cascade-2 pipeline
  • License: NVIDIA Open Model License

TaskTrove

  • Dataset Link
  • GitHub
  • Summary: 750K+ agentic task specifications in the Harbor format, designed for RL and evaluation of agentic coding/computer-use models. Companion to AgentTrove; tasks can be deployed via Harbor to update model behavior.
  • Data generation model: COL
  • License: Apache 2.0

SYNTHETIC-2-RL

  • Dataset Link
  • Blog
  • Summary: 156K RL-focused problems from the SYNTHETIC-2 collection with difficulty/reward annotations from Qwen3-32B, Qwen3-4B, DeepSeek-R1-0528-Qwen3-8B, and DeepSeek-R1-0528. Each row includes verification info suitable for rule-based reward training.
  • Data generation model: DeepSeek-R1-0528, Qwen3

UltraInteract_preference

  • Dataset Link
  • Paper
  • Summary: ~219K preference pairs (and up to 286K correct/incorrect nodes) organized as preference trees for multi-turn reasoning and tool-use trajectories. Supports both reward-model and DPO-style training.
  • Data generation model: GPT-4

RLSTACK

  • Dataset Link
  • Summary: 868K DPO-style preference pairs built from Stack Exchange dumps across 53 communities. Each example contains a prompt and paired chosen (highest-rated) / rejected (lowest-rated) answers.
  • Data generation model: N/A
  • License: CC BY 4.0

DPO Datasets

DPO Statistics

Project Links Org Nums Lang Summary
Magpie-Pro-DPO Magpie-Align/Magpie-Pro-DPO-100K-v0.1 Magpie-Align 100k English Preference pairs synthesized from aligned LLMs for DPO training.
Orca-DPO-Pairs HuggingFaceH4/orca_dpo_pairs |Intel/orca_dpo_pairs |argilla/distilabel-intel-orca-dpo-pairs HuggingFaceH4 / Intel / Argilla 12k / 6k English DPO preference pairs from GPT-4 Orca reasoning traces; Argilla cleaned version ~6k.
UltraFeedback-Binarized HuggingFaceH4/ultrafeedback_binarized HuggingFaceH4 61k-230k English Binarized UltraFeedback into chosen/rejected pairs for DPO; decontaminated variants available.
DPO-Mix-7K argilla/dpo-mix-7k Argilla 7.5k English Curated cocktail of Capybara, Intel Orca, and UltraFeedback high-quality DPO pairs.
Llama3-UltraFeedback-ArmoRM princeton-nlp/llama3-ultrafeedback-armorm Princeton NLP 60k English 5 Llama-3-SFT responses per prompt ranked by ArmoRM for advanced preference research.
ChatML-DPO-Pairs mlabonne/chatml-dpo-pairs Maxime Labonne 12k English ChatML-formatted Intel Orca DPO pairs (ChatGPT chosen vs Llama-2 rejected).
Magpie-Air-DPO Magpie-Align/Magpie-Air-DPO-100K-v0.1 Magpie-Align 100k English Synthetic DPO preference pairs from Llama-3-8B-Instruct (Magpie-Air).
Tulu 3 Preference allenai/tulu-3-pref-personas-instruction-following |allenai/tulu-3-pref-mixture Ai2 20k-200k English Preference data used in Tulu 3 post-training pipeline.
RLSTACK H-D-T/RLSTACK Hive-Digital-Technologies 868k English Stack Exchange DPO-style preference pairs (chosen/rejected answers).
UltraInteract_preference openbmb/UltraInteract_preference OpenBMB (Tsinghua) ~219k English Preference trees for multi-turn reasoning and tool-use trajectories.
Ling-Coder-DPO inclusionAI/Ling-Coder-DPO inclusionAI / Codefuse 253k English Code-specific DPO pairs curated from code_contests using test-pass, PPL, and reward-model signals.
DAPO-Math-17k BytedTsinghua-SIA/DAPO-Math-17k ByteDance / Tsinghua SIA 1.79M English Competition-level math problems with reward metadata for RL/DPO training.

DPO Dataset Details

Orca-DPO-Pairs

~12k DPO preference pairs derived from GPT-4 augmented Orca/FLAN reasoning traces. Intel released a widely used variant; Argilla later cleaned it to ~6k high-quality pairs, swapping/rejecting mislabeled examples.

UltraFeedback-Binarized

61k–230k preference pairs obtained by binarizing UltraFeedback scores into chosen/rejected responses. A standard DPO dataset used in Zephyr, Tulu 3, and many open alignment recipes.

DPO-Mix-7K

A small, curated cocktail (~7.5k pairs) mixing high-scoring chosen responses from Capybara, Intel Orca, and UltraFeedback. Designed for efficient, high-quality DPO training.

Llama3-UltraFeedback-ArmoRM

~60k prompts each with five Llama-3-SFT responses ranked by the ArmoRM reward model. Supports advanced preference optimization research beyond binary pairs.

ChatML-DPO-Pairs

~12k ChatML-formatted preference pairs based on Intel/orca_dpo_pairs, with ChatGPT responses as chosen and Llama-2-13b-chat responses as rejected.

Magpie-Air-DPO

~100k synthetic DPO preference pairs generated from Llama-3-8B-Instruct using the Magpie-Air pipeline.

Tulu 3 Preference

Ai2's Tulu 3 preference data (~20k–200k) including persona-based instruction-following preferences and a full preference mixture for open post-training.

RLSTACK

Cross-reference: see the full RLSTACK entry in the RLHF / Preference Datasets section. It is also used as a DPO-style preference dataset.

UltraInteract_preference

Cross-reference: see the full UltraInteract_preference entry in the RLHF / Preference Datasets section. It is also used for DPO training.

Ling-Coder-DPO

253K code-specific preference pairs derived from code_contests. Positive/negative samples are selected using code test-case pass rates, perplexity distribution, and reward model scores. Used to train Ling-Coder-Lite.

DAPO-Math-17k

1.79M competition-level math problems with reward-model metadata (ground_truth, evaluation style). Intended for RL and preference optimization such as GRPO/DPO; part of the DAPO collection.

ModelScope Chinese Datasets

A curated set of Chinese instruction-tuning and agent datasets available on ModelScope. These are also listed in the Multilingual & Chinese tables above; this section provides quick access to ModelScope links.

Dataset ModelScope Link Size Description
COIG-CQIA m-a-p/COIG-CQIA 48k High-quality Chinese instructions following the "Quality is All You Need" philosophy.
MSAgent / MSAgent-Bench iic/MSAgent-Bench 600k+ Chinese/English tool-use and agent training data.
DeepCtrl-SFT deepctrl/deepctrl-sft-data 12M Safe Chinese/English general-domain SFT data (~3B tokens).
Infinity Instruct AI-ModelScope/Infinity-Instruct 7M+ Large-scale Chinese/English instruction collection by BAAI/FlagOpen.
Alpaca-GPT4 Chinese AI-ModelScope/alpaca-gpt4-data-zh 52k Chinese Alpaca instruction data generated by GPT-4.
HC3-Chinese simpleai/HC3-Chinese 39,781 Chinese human vs ChatGPT response comparison corpus.
OpenHermes 2.5 swift/OpenHermes-2.5 ~1M High-quality general instruction mixture.
Magpie-Qwen2-Pro-200K-Chinese HuggingFace 200k Chinese alignment synthesis data based on Qwen2.
Chinese-DeepSeek-R1-Distill-110k-SFT liucong/Chinese-DeepSeek-R1-Distill-data-110k 110k Chinese reasoning SFT data distilled from DeepSeek-R1-671B.
Seq-Monkey ddzhu123/seq-monkey 10B tokens Mobvoi's Chinese general pre-training corpus.
smoltalk-chinese openscsg/smoltalk-chinese 700k+ Chinese general instruction-tuning data.
OpenThoughts3-1.2M open-thoughts/OpenThoughts3-1.2M 1.2M OpenThoughts 3 reasoning dataset (math / code / science).

Tip: You can use these datasets directly with the ms-swift framework.

Datasets without license information

alespalla/chatbot_instruction_prompts

  • Summary: A compilation of tatsu-lab/alpaca ,Dahoas/instruct-human-assistant-prompt ,allenai/prosocial-dialog
  • Data generation model: N/A
  • paper: N/A
  • Cost: N/A

Contributing

Our purpose is to make this repo even better. If you are interested in contributing, please refer to HERE for instructions in contribution.

Contribution Template

Append the new project at the end of file:

[{Project-name}/{Dataset-name}](https://github.com/link/to/project)

- [paper/project link](link)
- [dataset link](link)
- Related work: (if applicable)

Some introductions ...

License

Awesome-Prompt-Dataset is released under the Apache 2.0 license.

Reference

About

A collection of awesome-prompt-datasets, awesome-instruction-dataset, to train ChatLLM such as chatgpt 收录各种各样的指令数据集, 用于训练 ChatLLM 模型。

jianzhnie.github.io/llmtech

Topics

chatgptdatasetsinstructionllamallmpromptsself-instruct

Resources

Readme
Apache-2.0 license
Activity

Stars

743 stars

Watchers

5 watching

Forks

43 forks
Report repository

Releases

Packages

Used by

Contributors