Resources

/

Post-Training

Post-Training Hub

Guides and data stories on the post-training stages that need human judgment most: supervised fine-tuning, RLHF and verifiable rewards. Learn who should write the data, how to keep many experts consistent, and how expert work gets from review into a training pipeline.

Post-Training Hub

The long read · 1 of 3

How to Build an SFT Dataset with Domain Experts: A Supervised Fine-Tuning Guide

How to Build an SFT Dataset with Domain Experts: A Supervised Fine-Tuning Guide

Supervised fine-tuning uses thousands of examples, so each expert-written answer shapes the model. How to qualify writers, set guidelines and length limits, review answers and format the SFT dataset.

Read the guide

Key takeaways

  • Published SFT pipelines use thousands of examples; filtered sets beat larger unfiltered ones.
  • Set answer length in the guideline, because LLM judges reward longer answers.
  • Ask experts to stay within facts the model can already support.
  • Write the acceptance rule before labeling, and never let writers approve their own answers.

The long read · 2 of 3

Reward Models Explained: Preference Data, Rubric Rewards and How They Fail

Reward Models Explained: Preference Data, Rubric Rewards and How They Fail

A reward model turns human judgment into a training score. Learn how preference and rubric reward models work in law, medicine and insurance, and how they fail.

Read the guide

Key takeaways

  • Benchmark accuracy only loosely predicts the policy a reward model trains.
  • Reward models turn annotator disagreement into confident score gaps.
  • Human rubrics beat model-written ones by 22.1 to 28.3 points (RubricBench).
  • Physician-to-physician agreement on HealthBench ranged from 55% to 75%.
  • We found no published insurance reward model study.

The long read · 3 of 3

GRPO Explained: Group Relative Policy Optimization and the Data It Consumes

GRPO Explained: Group Relative Policy Optimization and the Data It Consumes

GRPO is the reinforcement learning algorithm behind DeepSeek-R1, Qwen3 and DeepSeek-V3.2. This guide explains how group relative policy optimization scores multiple responses against each other, and why your prompts and your reward model decide whether it learns.

Read the guide

Key takeaways

  • GRPO drops PPO's critic and uses the group's mean score as the baseline.
  • If every sampled answer gets the same score, that prompt produces zero learning signal.
  • Qwen3's reasoning RL used 3,995 selected query–verifier pairs.
  • Scorer errors land directly in the advantage, because GRPO builds its baseline from the scores themselves.

Data stories

Quick answers

Short answers to the questions teams ask us most in this area.

What is post-training, and how is it different from fine-tuning?

What is supervised fine-tuning data, and who should write it?

What is RLHF, and what do human experts do in it?

What is RLVR, and where do domain experts fit in?

Build post-training data with the experts who know the answer