Resources
/
Post-Training
Guides and data stories on the post-training stages that need human judgment most: supervised fine-tuning, RLHF and verifiable rewards. Learn who should write the data, how to keep many experts consistent, and how expert work gets from review into a training pipeline.
01
02
Short answers to the questions teams ask us most in this area.
Post-training is everything done to a pretrained model to make it useful for a task, and fine-tuning is one part of it. A typical recipe combines supervised fine-tuning on expert-written examples, preference training such as RLHF or DPO, and reinforcement learning with verifiable rewards, the three stages of AI2's Tülu 3. All three depend on human judgment: people write the target answers, choose between responses and decide what counts as correct. In Kili, that work is done by your own specialists, in projects with review steps and agreement scores built in.
Supervised fine-tuning (SFT) data is a set of prompts paired with the answers you want a model to give, and the model learns to imitate those answers closely. That is why the author matters: an answer that is plausible but not quite right teaches the model to be plausible. In Kili, a specialist can start from the model's own draft, imported as a prediction, and rewrite it in a transcription job. The draft and the expert's version both stay on the asset and can be retrieved separately through the Python SDK.
Reinforcement learning from human feedback (RLHF) trains a model on people's judgments about which of its responses is better. Annotators compare responses to the same prompt, rate how strongly they prefer one, and sometimes edit the better one further; Meta's Llama 3 team ranked those edited answers above both originals. In Kili's LLM projects, which are currently experimental, specialists rank responses from two or more models, rate individual turns and whole conversations, and provide improved answers.
Reinforcement learning with verifiable rewards (RLVR) trains a model on problems whose answers a program can check, such as a numeric result or a proof that compiles, so the reward needs no person at training time. The human work moves earlier: experts write the problems, the reference answers and the checks. In Kili, a Python plugin can run your checks each time an expert submits, add an issue and send the asset back when a check fails. A leading AI research lab used Kili to formalize about 2,500 competition math problems in Lean 4, with peer review and compiler verification on every statement.
Newsletter
New guides and data stories, by email
New posts from the AI Evaluation, Post-Training and Data Labeling hubs, sent when they go live.