Resources
/
Data Labeling
Guides on building labeled data that general annotators can’t: when to bring in domain experts, how to write guidelines and ontologies that hold up, where model pre-labels help, and how review, agreement checks and routing keep every label defensible.
01
02
Short answers to the questions teams ask us most in this area.
Expert data labeling is labeling done by people qualified to make the call the label records, such as clinicians, imagery analysts or quality engineers. You need them wherever the correct class depends on domain knowledge rather than on what is visible, for example whether a finding in a clinical note is negated, or whether a mark on a casting is a defect or a tool mark. In Kili, each asset can be assigned to a named specialist, and consensus, honeypot and review steps measure whether their labels hold up.
Labeling documents for intelligent document processing (IDP) means marking each field the model should find, reading its value, and recording which document type it came from, on the same kind of scans the model will see in production. In Kili, drawing a bounding box over a field on an image or PDF fills in its text through OCR, entirely in the annotator's browser, and the labeler corrects it where needed. Relation jobs link each value to its field, a classification job records the document type, and consensus on the OCR job shows where specialists read a field differently.
Label multispectral satellite imagery with the bands that show the feature on screen, because features such as crop stress or standing water can be much clearer in near-infrared than in true colour. In Kili, geospatial projects take GeoTIFF and Cloud-Optimized GeoTIFF files, let the labeler build composites such as NDVI or false colour from any combination of bands, and offer bounding boxes, polygons, points, lines and semantic segmentation on the map. Labels are exported through the Python SDK in GeoJSON, with coordinates as longitude and latitude.
Consensus measures annotators against each other, and honeypot measures them against a reference you supply. Use consensus when no ground truth exists yet and the question is whether your criteria are clear enough for two specialists to reach the same call. Use honeypot when you already hold labels you trust and want to know whether a labeler is drifting from them.
Report
Newsletter
New guides and data stories, by email
New posts from the AI Evaluation, Post-Training and Data Labeling hubs, sent when they go live.