AI-enabled applications can transform an organization, but to create value, these applications must integrate into business processes and be intuitive for end users. Organizations often overlook the effort of operationalizing AI-enabled insights—one of the most challenging aspects of successfully deploying AI-enabled applications. At C3.ai, we have identified four key strategies to overcome this challenge:
- 1. Understand the users and target their needs
- 2. Streamline actions by integrating with existing systems
- 3. Build closed loop systems for continuous machine learning
- 4. Track business KPIs within the application to demonstrate value over time
Understanding the Challenge
What does it mean to operationalize an AI-enabled application?
Why is this hard? There are three main reasons why convincing people to take action based on AI-enabled predictions is challenging:
- Users may mistrust AI predictions: “People are not probability thinkers, but cause-effect thinkers,” declared Dr. Judea Pearl during a 2012 Cambridge University Press interview. His statement captures a critical issue for communicating in terms of probability, statistics, and data – it isn’t natural to most people. This challenge is so immense that since Pearl’s 2012 interview, this concept has been widely reported on by Harvard Business Review, Nate Silver, Daniel Kahneman, and the data science community, among others.
- Users suffer information and data overload: Living in a modern society, we are surrounded by data. Everything from our wearables to our cars to the stores in which we shop collects and shares data on a massive scale. While this has led to incredible digital transformation, it has also created a challenge for people to interpret and understand all of the information suddenly at their fingertips. Read this article from Medium if you need further convincing.
- Organizations are trapped by a reactive modus operandi: Modern companies often function in “fire-fighting” mode, constantly reacting to issues resulting from fast-paced workplaces and high volumes of work. In such an environment, it is a monumental effort to shift operations toward prioritizing future risk versus focusing on current issues. This type of change requires engaging the entire organization.
Strategies for overcoming these challenges and convincing slow-adopters to become AI advocates
Strategy 1: Understand the users and target their needs
Listening to customers sounds like a simple task but can be a complex process in reality. Users often don’t know what they want, and product development requires deciphering their needs. The first step is to identify potential end users of the application. This can be a challenge in and of itself – for C3 Predictive Maintenance, we may have users who are technicians, engineers, supply chain analysts, managers, or all of the above.
Designing one single application to meet so many different requirements means that the application should be efficient—we don’t want to over-load the user with unnecessary data, nor make entirely separate applications for every individual–and contain specific workflows that make sense for distinct roles. For example, a technician-specific dashboard may display an easy-to-access ranked list of equipment risks, along with recommended parts or interventions to address each risk, and a history of past work orders. While that same detail could be accessible to engineers or managers, those users will have a different dashboard for key information relevant to their daily needs.
This concept is shown in the “Information Efficiency Curve” illustration. Strategy 1 is all about figuring out what information is contained in the peak of user productivity for each type of user. This strategy mitigates the challenge of data overload. At C3.ai, we simplify AI insights and present them in an intuitive format that shows the drivers behind the AI-generated prediction, allowing users to understand and take action going forward.
It is critical to only show the optimal information up front and provide the details for a user looking to dive more deeply.
Strategy 2: Streamline actions by integrating with existing systems
The best way to successfully operationalize AI predictions is to automatically integrate with existing systems, like work order management systems (OMS). Auto-integration removes the natural human uncertainty about a machine-learning prediction. It also guarantees that the most value is being captured from the application.
For example, in the illustration below, a C3.ai AI-generated risk prediction can be used to automatically generate a work order in a third-party system. An operator can review the work order, open C3.ai’s recommended interventions, and take appropriate action to fix the issue. This integration helps guide the user to the correct action, rather than simply alerting them of a failure risk. It also helps to improve the machine learning model by providing closed-loop feedback on each prediction.
As these applications become more advanced, organizations can integrate many business operations elements to create recommended actions. For example, C3 Predictive Maintenance can integrate with supply chain planning to recommend ordering more parts, thereby avoiding a shortage. These streamlined workflows result in less downtime and more efficient operations.
Strategy 3: Build closed-loop systems for continuous machine learning
The above illustration also shows the strategy for closed-loop learning. By tracking a maintenance action back into source data systems, AI algorithms can continue to improve for future predictions. For example, if the feedback loop contains the information ‘What action was taken in response to the risk prediction?’ the machine learning model will be able to learn ‘What is the best recommended action in response to the risk score?’ This closed loop means the predictions get more accurate over time, and to become richer in detail.
This closed-loop feedback system may seem simple in theory but is often overlooked in practice. Though careful planning and customer workshops, we make sure that the closed loop systems include the critical information to track downstream actions and impacts.
Strategy 4: Measure business KPIs within the application to demonstrate value over time
The last step for successfully operationalizing an AI application is measuring key performance indicators to demonstrate value. Unless these metrics are defined in advance, it can be hard to quantify the benefits of moving from reactive to proactive operations. It is even harder to communicate machine learning concepts like ROC and AUC to a business audience, so it is more effective to use terms accepted by the organization. Of course, machine learning scores and performance should also be closely monitored, but that is the responsibility of the data scientists.
To achieve this result, we build value calculations for relevant KPIs into the application dashboard and display the impact over time. For example, in a C3 Predictive Maintenance application for oil & gas, the dashboard screen may show: increase in hydrocarbon volume produced over time, number of failure events avoided, and decrease in downtime. Communicating in terms of these operations-based metrics generates trust in the application and demonstrates the value of switching to proactive versus reactive operations.
These are the four key strategies that we use at C3.ai to help customers take action based on AI: first, understand the users and their needs; second, integrate the application into existing systems to streamline the workflow; third, create a closed-loop data flow for continuous machine learning; and fourth, communicate value in terms understood by the user. These four strategies have helped us successfully enable customers to take action with AI in aerospace, oil & gas, manufacturing, and beyond.
Lila Fridley at Girl Geek X
January 17, 2019
Watch Lila share her product management insights and experience working on AI predictive maintenance in aerospace at Girl Geek X.