I had the opportunity recently to speak at the recent Data Driven Drilling & Production Conference in Houston and spend time with some of the industry’s leading digital and data science practitioners. Some things have clearly changed since I left oil & gas six years ago, but some haven’t. The biggest thing that has changed is that digital has become important again. For example, one Fortune 10 oil & gas company’s CEO has made digital one of his top three priorities and almost every company is doing data science.
A new model-driven AI architecture to solve data integration and scaling challenges
One thing that hasn’t changed: almost everyone is doing data science like I did analytics eight years ago. The dominant analytics paradigm in 2010 was building analytic and workflow solutions using service-oriented architectures to overcome the challenges of data integration, and then operationalizing the analytics and scaling the solutions across multiple locations. Based on the technology we had, this was the best approach. But it barely scratched the surface of what we needed to do to increase production and reduce costs. And, to a large extent, we were building islands of analytics. Since then, we’ve seen cloud services continue to expand their capabilities, growth in the importance of data science and machine learning, and lots of talk about IoT and the industrial internet. And we’ve seen incrementally better solutions. But we haven’t seen a fundamentally new analytics paradigm.
Like the oil industry, most industry-focused tech companies are investing in better ways to build islands of analytics. Cloud services provide not just technology services but a way to manage and orchestrate them. Big data technologies are widely available as open source or commercial distributions. Today, it’s much easier to build islands of analytics than it was in 2010. But the challenges of data integration, operationalizing data science, and scaling solutions across diverse operations remain. Almost every presentation at the Data Driven Drilling & Production conference mentioned these challenges.
A New Paradigm Emerges
I think that a new paradigm is emerging. And when a new paradigm emerges, what was once hard becomes easy. But until you believe that has happened, you can’t see it.
I’ve been fortunate to have spent the last eight years investigating other industries and spending considerable time with emerging technology players. I see a new paradigm emerging. It combines today’s technology trends—cloud services, machine learning, IoT, and big data technologies—and leverages some things we’ve been working on for decades, like making metadata work. I’ve been calling it a “new operational architecture” because it addresses the three challenges of data integration, operationalizing data science, and scaling solutions.
Overcoming the Challenges of Integration
So, what’s different about this paradigm and how does it overcome these three challenges? First, it is built on top of the technologies developed in the last 15 years –cloud and big data. Second, it is based on a model-driven architecture, an approach introduced in 2001 as a successor to object-oriented architecture. Finally, it includes a platform for management and orchestration of data and technology services.
All of these together give a company the ability to:
- Solve the data integration and scaling challenges by creating a unified, federated cloud data image of all a company’s data in a single data model, updated in near real time.
- Solve the operationalizing challenge by having data scientists and developers working on the same data set in the same platform simultaneously to develop and deploy intelligent analytic and workflow applications.
- Solve the scaling challenge by enabling companies to develop and deploy solutions incrementally on the unified data image, making them easy to introduce into operations and easy to extend to other oil and gas fields.
Today, companies like C3.ai are providing the platform that makes this possible. And there are a few companies I’ve spoken with in the energy industry like Origin, Shell, and Engie that have spent time to understand the new paradigm. They believe it, and now they see the opportunity to drive step changes in performance and cost structure. I believe these are the companies that will best compete in today’s ‘lower for longer’ price environment.
