Cleanlab In the News
Featured publications by our team
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
Curtis Northcutt, Anish Athalye, and Jonas Mueller. 35th Conference on Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2021
Confident Learning: Estimating Uncertainty in Dataset Labels
Curtis Northcutt, Lu Jiang, and Isaac Chuang. Journal of Artificial Intelligence Research (JAIR), 2021
Detecting Errors in a Numerical Response via any Regression Model
Hang Zhou, Jonas Mueller, Mayank Kumar, Jane-Ling Wang, and Jing Lei. Journal of Data-centric Machine Learning Research (DMLR), 2024
ActiveLab: Active Learning with Re-Labeling by Multiple Annotators
Hui Wen Goh and Jonas Mueller. ICLR Workshop on Trustworthy ML, 2023
CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators
Hui Wen Goh, Ulyana Tkachenko, and Jonas Mueller. NeurIPS 2022 Human in the Loop Learning Workshop
ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data
Ulyana Tkachenko, Aditya Thyagarajan, and Jonas Mueller. ICML Workshop on Data-centric Machine Learning, 2023
Detecting Dataset Drift and Non-IID Sampling via k-Nearest Neighbors
Jesse Cummings, Elías Snorrason, and Jonas Mueller. ICML Workshop on Data-centric Machine Learning, 2023
Estimating label quality and errors in semantic segmentation data via any model
Vedang Lad and Jonas Mueller. ICML Workshop on Data-centric Machine Learning, 2023
DataPerf: Benchmarks for Data-Centric AI Development
Mazumder et al.. Advances in Neural Information Processing Systems (NeurIPS), 2023
Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels
Curtis Northcutt, Tailin Wu, and Isaac Chuang. 33rd Conference on Uncertainty in Artificial Intelligence (UAI 2017)
Additional publications by our team
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
Anish Athalye, Nicholas Carlini, and David Wagner. 35th International Conference on Machine Learning (ICML), 2018
EgoCom: A Multi-person Multi-modal Egocentric Communications Dataset
Curtis Northcutt, Zha Shengxin, Steven Lovegrove, and Richard Newcombe. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
Time-Varying Propensity Score to Bridge the Gap between the Past and Present
Rasool Fakoor, Jonas Mueller, Zachary Lipton, Pratik Chaudhari, and Alex Smola. International Conference on Learning Representations (ICLR), 2024
Adaptive Interest for Emphatic Reinforcement Learning
Martin Klissarov, Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Taesup Kim, and Alex Smola. Advances in Neural Information Processing Systems (NeurIPS), 2022
Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features
Jiuhai Chen, Jonas Mueller, Vassilis Ioannidis, Soji Adeshina, Yangkun Wang, Tom Goldstein, and David Wipf. International Conference on Learning Representations (ICLR), 2022
Deep learning for the partially linear Cox model
Qixian Zhong, Jonas Mueller, and Jane-Ling Wang. Annals of Statistics, 2022
Flexible Model Aggregation for Quantile Regression
Rasool Fakoor, Taesup Kim, Jonas Mueller, Alexander Smola, and Ryan Tibshirani. Journal of Machine Learning Research, 2023
Benchmarking Multimodal AutoML for Tabular Data with Text Fields
Xingjian Shi, Jonas Mueller, Nick Erickson, Mu Li, and Alex Smola. 35th Conference on Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2021
Overinterpretation reveals image classification model pathologies
Brandon Carter, Siddhartha Jain, Jonas Mueller, and David Gifford. Advances in Neural Information Processing Systems (NeurIPS), 2021
Deep Extended Hazard Models for Survival Analysis
Qixian Zhong, Jonas Mueller, and Jane-Ling Wang. Advances in Neural Information Processing Systems (NeurIPS), 2021
Continuous Doubly Constrained Batch Reinforcement Learning
Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Pratik Chaudhari, and Alex Smola. Advances in Neural Information Processing Systems (NeurIPS), 2021
Graph Neural Networks Formed via Layer-wise Ensembles of Heterogeneous Base Models
Jiuhai Chen, Jonas Mueller, Vassilis Ioannidis, Tom Goldstein, and David Wipf. Transactions on Machine Learning Research (TMLR), 2024
Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challenges
Massimo Caccia, Jonas Mueller, Taesup Kim, Laurent Charlin, and Rasool Fakoor. Conference on Lifelong Learning Agents (CoLLAs), 2023
Verifying Hardware Security Modules with Information-Preserving Refinement
Anish Athalye, M. Frans Kaashoek, and Nickolai Zeldovich. 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2022
Notary: A Device for Secure Transaction Approval
Anish Athalye, Adam Belay, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich. 27th ACM Symposium on Operating Systems Principles (SOSP), 2019
Synthesizing Robust Adversarial Examples
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 35th International Conference on Machine Learning (ICML), 2018
Black-box Adversarial Attacks with Limited Queries and Information
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 35th International Conference on Machine Learning (ICML), 2018
Identifying Incorrect Annotations in Multi-Label Classification Data
Aditya Thyagarajan, Elías Snorrason, Curtis Northcutt, and Jonas Mueller. ICLR Workshop on Trustworthy ML, 2023
Detecting Label Errors in Token Classification Data
Wei-Chen (Eric) Wang and Jonas Mueller. NeurIPS Workshop on Interactive Learning for Natural Language Processing (InterNLP), 2022
Back to the Basics: Revisiting Out-of-Distribution Detection Baselines
Johnson Kuan and Jonas Mueller. ICML Workshop on Principles of Distribution Shift, 2022
Model-Agnostic Label Quality Scoring to Detect Real-World Label Errors
Johnson Kuan and Jonas Mueller. ICML DataPerf Workshop, 2022
Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness
Jiuhai Chen and Jonas Mueller. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Automated Data Curation for Robust Language Model Fine-Tuning
Jiuhai Chen and Jonas Mueller. Preprint, 2024