Data integration is one of the critical first steps when building any AI application. And although there are multiple methods to choose from when starting this process, data virtualization is what allows organizations to accelerate the application development and deployment process.
How the C3 AI Platform Helps Realize Value from Data Quickly
The C3 AI Platform was designed to give customers a fast and flexible way to put their data to work for them, delivering AI insights in real time for advanced programs such as predictive maintenance, fraud detection, and demand forecasting. When establishing these initiatives, most companies already have robust databases and technology investments that they are looking to quickly gain more value from. Data virtualization is what enables organizations to leverage their existing data sources in a flexible and efficient way for AI and analytics applications.
On the C3 AI Platform, data virtualization is when external data is read from source systems and represented on the platform on demand. It provides a centralized and modern approach to accessing and managing data without physically storing it. A virtual layer is created between data sources and consumers, allowing organizations to access and manage their data without copying or moving it from its original location.
To speed up this process even further, the C3 AI Platform offers a library of over 200 pre-built data connectors that link and integrate legacy systems and databases. These connectors allow developers to easily take advantage of the data virtualization capabilities of the platform. The library contains connectors to various source systems such as Postgres, Oracle, SAP, Apache HBase, Apache Impala, Snowflake, and Cassandra. And if a data connector to a specific source does not yet exist, C3 AI can quickly build a custom one.
Key Benefits of Data Virtualization
- Enables seamless integration of data from multiple sources, without the need to physically move or copy the data. This can significantly reduce the time and cost involved in data integration, while also reducing the risk of data errors or loss.
- Allows organizations to create a single, unified view of their data, regardless of where it is stored. This provides a single source of truth for analytics, reporting and decision making, improving the accuracy and speed of data-driven insights.
- Enables organizations to scale and adapt their data to meet changing business needs, providing greater flexibility and agility.
Using Data Virtualization to Build a Predictive Maintenance Application
C3 AI has leveraged the power of virtualization to develop a predictive maintenance solution for a large manufacturing customer. This solution consumes virtualized data from multiple source systems like OSI PI and AWS Kinesis streams to provide a comprehensive view of the organization’s operational data. The data is then analyzed using machine learning (ML) algorithms to identify patterns and trends that are used to predict potential failures and schedule maintenance activities before they occur.
With over 220 assets sending 400k data points per day, the ML models generate 2,600 predictions hourly. Over 1200 actionable insights have been generated over the last year allowing the customer to take proactive steps to address issues and improve the overall health of their assets or equipment.
The Value of External Types
On the platform, external types model the physical structure of the external data, such as a table or a view, and directly retrieve the data from the source system. Unlike data ingestion, the data is not stored in any C3 AI data store. The platform’s APIs that access and evaluate data are identical regardless of whether the underlying data is ingested or virtualized. At runtime, the C3 AI Platform will orchestrate data retrieval from the source system, store the records in memory for computation, and then drop the data from memory once the computation is complete.
Put simply, external types allow for an efficient and direct connection to the original data sources. This means businesses can get real-time insights without the overhead of storing duplicate data, ensuring they always have the most up-to-date information at their fingertips.
Design Considerations: How C3 AI Builds an AI Application for Predictive Maintenance
- Latency in accessing virtualized data: In the context of predictive maintenance, timely data access is essential. The delay in retrieving virtualized data as compared to directly accessed persisted data can affect real-time predictive analyses. Design considerations should account for this latency to ensure maintenance predictions remain timely.
- Frequency of accessing virtualized data: Continuously accessing virtualized data for real-time insights can potentially strain the source systems. For predictive maintenance, it’s crucial to strike a balance to avoid overwhelming the original data source while still getting frequent updates.
- Virtualization layer abstraction: One of the major advantages for developers building a predictive maintenance application on the C3 AI Platform is not having to be concerned with where the data resides. The virtualization layer’s abstraction offers ease of development by focusing on the application logic rather than data storage intricacies.
- Advanced C3 AI Type System concepts: The C3 AI Type System is an abstraction layer that allows users to connect C3 AI Platform components, including infrastructure and services, to each other without a full integration process. To connect a new component to the C3 AI Platform, users can select the corresponding C3 AI Type that contains the data object, methods, application logic, and ML classifiers needed for a seamless integration. And although certain C3 AI Types including TimedValue*, TimedRelations*, and TimedInterval* offer advanced functionalities, they are not supported for external types. This might limit the granularity of time-bound analyses for maintenance predictions but can be circumvented using alternative data modeling techniques.
- Stored calculated fields: Predictive maintenance applications might rely on calculated fields to derive insights. Since these are not supported on external types, the design needs to accommodate pre-calculations or alternative methods to derive the same insights. This might mean some additional logic on the application side or reliance on the source system’s capabilities.
There are a few other considerations to keep in mind like how to best optimize storage. Choosing to optimize either through normalization or denormalization might not be directly relevant to the predictive maintenance application, especially when relying heavily on external types, but they’re essential when considering a hybrid approach combining both ingested and virtualized data.
Solution Architecture
Applying Data Virtualization Across Applications
Data virtualization is not just an innovation, it’s a strategic tool. The C3 AI Platform exemplifies this by harnessing data virtualization to facilitate development of diverse applications, enhancing their efficiency, adaptability, and near real-time capabilities. Let’s delve deeper into some standout applications:
- Supply Chain Optimization: In today’s globalized economy, supply chains are vast and intricate. By virtualizing data from varied sources — ranging from production metrics to logistics tracking, to market trends — C3 AI empowers businesses with a panoramic view of their operations. This 360-degree visibility translates into actionable insights, enabling companies to preempt bottlenecks, streamline operations, and adapt to market dynamics on the fly, ensuring an optimized and agile value chain.
- Customer Analytics: In the age of digital transformation, understanding your customer is paramount. C3 AI harnesses data virtualization to amalgamate data from diverse touchpoints—be it sales, customer interactions, or marketing campaigns. The result? A comprehensive understanding of customer behavior patterns and preferences. With such deep insights, businesses can craft tailor-made experiences, promotions, and products, ensuring customer satisfaction and driving loyalty.
- Fraud Detection: Financial fraud is a pervasive issue that evolves continually, making its detection a moving target. C3 AI’s approach to this challenge is dynamic. By virtualizing and analyzing data from various sources — transactions, user behavior, demographic details, and more — the platform can detect anomalous patterns and potential fraud in real-time. This proactive approach not only protects businesses from potential losses but also fosters trust among their user base, an invaluable asset in the digital age.
Harnessing the power of data virtualization, C3 AI offers businesses a formidable arsenal to tackle contemporary challenges and capitalize on emerging opportunities.
Learn more about the C3 AI Platform and explore all its advanced capabilities, including data virtualization, that accelerate the development of robust AI applications.
