HOW IT WORKS
More Automated Quality Assurance
Provide automated quality assurance for data annotation teams working across diverse applications (speech transcription, autonomous vehicles, industrial quality control, image segmentation, object detection, intent recognition, entity recognition, content moderation, document intelligence, LLM evaluation and RLHF, and more).
Assessing Multiple Data Annotators
Use our CrowdLab system to analyze data labeled by multiple annotators and estimate: a consensus label for each instance that aggregates the individual annotations; a quality score for each consensus label to gauge confidence that it is the correct choice; and a quality score for each annotator to quantify the overall correctness of their labels.
Built with AI
Use our ActiveLab system (active learning with relabeling) to efficiently collect new data labels for training accurate models. Obtain reliable data labels even with (multiple) imperfect annotators. Only ask for labels that will significantly improve your model.
More Cost-Effective Workflows
Use least expensive data provider to first obtain noisy labels, and then ask more expensive provider (or in-house experts) to review select examples flagged by Cleanlab.
Compare Data Quality of Sources
Compare quality of different data providers and data sources. Listen to a discussion on this topic in the Weights & Biases podcast.
Resources and Tutorials
Videos on using Cleanlab Studio to find and fix incorrect labels for: text annotation or metadata, image annotation or metadata, and data tables.
CASE STUDIES
Google’s mission is to organize the world’s information and make it universally accessible and useful.
Google used Cleanlab to find and fix label errors in millions of speech samples across different languages, to quantify annotator accuracy, and provide clean data for training speech models.
1M+
speech samples processed
“Cleanlab is well-designed, scalable and theoretically grounded: it accurately finds data errors, even on well-known and established datasets [and] is now one of my go-to libraries for dataset cleanup.”
Senior Software Engineer at Google
Gavagai provides multilingual text analytics for customer insights. Analyzing reviews, surveys, call transcripts, support tickets, and social media, their platform helps discover, track, and act on customer data to improve Customer Experience.
Gavagai relies on labeled data to train our models, and the quality of the data is paramount when it comes to creating machine learning models that can produce business value for our customers. Cleanlab allowed the Gavagai team to upload a dataset and obtain a ranked list of all the potential label issues in the data and assess and fix them in the GUI.
“Cleanlab Studio is a very effective solution to calm my nerves when it comes to label noise [and] should be a go-to tool in every ML practitioners toolbox!”
Head of Data Science at Gavagai
BBVA is one of the largest financial institutions in the world. With a strong presence in multiple countries, BBVA offers a wide array of banking and financial services to individuals, businesses, and institutions.
BBVA used Cleanlab in an update of one of the functionalities offered by the BBVA app: the categorization of financial transactions. These categories allow users to group their transactions to better control their income and expenses, and to understand the overall evolution of their finances. The BBVA team also used AL [Active Learning] in combination with Cleanlab to reduce the impact of having different subcategories for similar transactions.
In addition, Cleanlab aided BBVA in reducing the uncertainty of noise in tags. This process enabled them to train their model, update the training set, and optimize its performance. The goal was to reduce the number of labeled transactions and make the model more efficient, requiring less time and dedication and allowing data scientists to focus on tasks that generate greater value for customers and organizations.
98%
reduction in required number of labeled training transactions
28%
improvement in ML model accuracy (with no change in existing modeling code)
“CleanLab helped us reduce the uncertainty of noise in the tags. […] This allows data scientists to focus on tasks that generate greater value for customers and organizations.”
Expert Data Scientist at BBVA
