Large enterprises are increasingly aware of the reputational, legal and financial risks posed by employing machine learning. That’s where Arthur.ai, a machine learning model monitoring company, comes in. It provides a layer of performance monitoring, explainability and algorithmic bias detection, to help clients continuously detect, diagnose, and fix issues that arise. The New York City-based startup was included in CBInsight’s just released “AI 100: The AI Startups Redefining Industries” list. It is the second year in a row that the three-year-old company has made the list. Its customers include health insurance provider Humana, the U.S. Air Force and fintech Truebill.
As more organizations scale their AI adoption, they’re discovering a need for greater visibility into how their models are behaving and the means to take action when models aren’t performing at peak, says co-founder and CEO Adam Wenchel. Data is sometimes trained in a way that is different from real world conditions, he says. Other times a seemingly inconsequential change can lead machine learning models to make decisions that are all wrong. Or, the world shifts gradually and the model slips from a performance rating of 98.8 to, 98.7, to 98.6, and the continuing deterioration goes unnoticed. “Such small incremental declines can be hugely impactful and cost large enterprises hundreds of millions of dollars,” he says.
Companies can be held legally liable if they don’t make decisions made by machine learning models that impact humans explainable. And, if a problem arises and machine learning models used by companies are not transparent it can lead to reputational ruin. The same is true if companies fail to ensure equal access to outcomes in areas like healthcare or credit. Some 53 different models have been developed by academics to measure fairness, says Wenchel. Arthur.ai says it supports the most relevant ones to ensure that machine learning models in use by its clients meet thresholds for being reasonably fair. One of the things that make the startup unique is that in addition to alerting clients of any unfairness that might have crept into their machine learning models it is able to help clients actively fix the problem by adjusting training data and continuing to monitor the model under real-world conditions, Wenchel says.
Wenchel previously held the title of vice-president of AI and Data Innovation at Capital One, an American bank holding company specializing in credit cards, auto loans, banking, and savings accounts. He is credited with founding and leading Capital One’s Center for Machine Learning, catalyzing the adoption of machine learning in high value areas such as credit, user experience, cybersecurity, marketing and customer acquisition, fighting fraud and financial crimes, and automating finance and back-office operations. Wenchel says he founded Arthur.ai because when he was implementing machine learning at Capital One, he knew that he was impacting people’s lives and impacting revenues and that there were risks associated with that. “It kept me up at night in 2016 because there weren’t any monitoring solutions for this back then,” he says. “I saw a huge need.”
Client health insurance provider Humana is using Arthur to monitor and mitigate bias in its preventative care models, to try and ensure fair and explainable outcomes. Client Truebill is using it to monitor the performance of its customer lifetime value and customer churn models in order to catch and fix data drift before serious problems arise. The U.S. Air Force uses Arthur.ai’s technology to monitor machine learning models optimizing its supply chain.
The company, which has raised $18.3 million in venture capital, is expanding aggressively and recently added several large European clients, Wenchel says.
Arthur.ai competes with tools from cloud providers like SageMaker Clarify from AWS as well as startups like DataRobot. It recently partnered with Algorithmia, a leader in machine learning operations and model management, in order to offer clients an even more complete control over their production AI, Wenchel says.
To access more of The Innovator’s Startup Of The Week articles click here.