Label Studio

Label Studio is an open-source platform for the work that has to happen when a model is not good enough on its own: labelling data across text, images, audio and video, reviewing outputs, and running human-in-the-loop workflows — including the newer variants where what is being labelled is an agent's trace or an LLM's output rather than a photo.

Its value comes from a dull property: the labelling interface is configurable, so a task that does not look like the last task does not require writing a new tool. That is why a general labelling platform beats a bespoke script once you have more than one project, and why teams that start with a spreadsheet tend to arrive here later. The two things that decide whether it is enough are both about people rather than software: whether the annotation guidelines are specific enough that two annotators agree, and whether the quality controls catch the annotator who is approving everything. Agreement between annotators is itself a measurement, and the tool can only record it, not create it.