HOW CLEANLAB CAN HELP
Ensure reliable analytics so you can make good financial decisions
HOW IT WORKS
Assess for Outliers and Anomalies
Use advanced AI to detect outliers in text, images, and data files (Excel, CSV, JSON). Improve modeling of transactions, money reserves, and clients/providers. Cleanlab Studio offers comprehensive anomaly diagnosis beyond basic stats and past trends.
Detect Fraud
Real-world training datasets often contain significant label noise, including overlooked cases of fraud. Cleanlab algorithms automatically detect and correct instances that are mislabeled, improving the reliability of fraud detection model training and evaluation.
Assess Risk
Cleanlab Studio enables data-centric AI to build accurate ML models for messy real-world tabular or text data. You can effortlessly harness AutoML for various data types, including text, image, and tabular formats (Excel, CSV, Json), allowing you to focus on the most important aspect: the data.
Aggregate Decisions and Predictions from Multiple Annatators
Cleanlab can analyze data annotated with multiple individuals’ decisions and estimate: A consensus decision for each instance that aggregates the individual decisions, a quality score for each consensus decision to gauge confidence that it is the correct choice; and a quality score for each annotator to quantify their overall skills.
Resources and Tutorials
Videos on using Cleanlab Studio to find and fix incorrect labels for: text data, tabular data, and image data.
CASE STUDY
BBVA is one of the largest financial institutions in the world. With a strong presence in multiple countries, BBVA offers a wide array of banking and financial services to individuals, businesses, and institutions.
BBVA used Cleanlab in an update of one of the functionalities offered by the BBVA app: the categorization of financial transactions. These categories allow users to group their transactions to better control their income and expenses, and to understand the overall evolution of their finances. The BBVA team also used AL [Active Learning] in combination with Cleanlab to reduce the impact of having different subcategories for similar transactions.
In addition, Cleanlab aided BBVA in reducing the uncertainty of noise in tags. This process enabled them to train their model, update the training set, and optimize its performance. The goal was to reduce the number of labeled transactions and make the model more efficient, requiring less time and dedication and allowing data scientists to focus on tasks that generate greater value for customers and organizations.
98%
reduction in required number of labeled training transactions
28%
improvement in ML model accuracy (with no change in existing modeling code)
“CleanLab helped us reduce the uncertainty of noise in the tags. […] This allows data scientists to focus on tasks that generate greater value for customers and organizations.”
Expert Data Scientist at BBVA
To automatically triage future customer requests, intent classification is a standard Machine Learning task in customer service applications that requires well-labeled data.
A customer at an online bank used Cleanlab on a dataset of customer requests (text) annotated with 10 different intents. Cleanlab Studio found over 5% of dataset labels were incorrect and detected out-of-scope queries (outliers) like “how much is 1 share of aapl” and “is android better than iphone.”
The dataset underwent customer analytics to discern the frequency of various customer requests and identify the most prevalent types, but inaccuracies stemmed from mislabeling and out-of-scope issues within the original dataset. A cleaned version from Cleanlab Studio yielded significantly more precise conclusions, resulting in a 16% reduction in prediction error without modifications to the existing LLM Transformer model or training code. Further improvements in accuracy were achieved by addressing additional data issues without altering the model.
5%
of dataset labels found to be incorrect
16%
reduction in prediction error
“Using Cleanlab Studio to auto-fix label issues in this dataset led to a 16% improvement in prediction error without altering the existing LLM Transformer model or training code.”
