The End of Integration

How Dataware Could Change Applications Forever

Through a series of blogs, webinars and a white paper Joe Hilleary shares these insights on data centralization approaches including dataware. 

As defined in previous posts, dataware is an emerging approach to data architecture that seeks to eliminate the need for data integration. This article lays out both the factors driving its adoption and the remaining hurdles between dataware and its ultimate goals.

Dataware is more than a product offering. It’s a radical reimagining of the relationship between data, applications, and analytics. Ultimately, the dataware approach seeks to upend decades of assumptions about how software should be designed in order to eliminate the burden of data integration and end data silos once and for all. It provides a powerful and encompassing vision of what the future could look like.

Instead of every application shipping with its own database--fragmenting organizations’ data from the very beginning--dataware proposes a shared data layer. In this framework, applications would simply provide sets of instructions. Just as today software relies on hardware to execute the physical tasks required to run programs, tomorrow it will rely on dataware to handle the data storage and management tasks. You don’t expect modern applications to ship with their own devices, so why should you expect them to ship with their own databases?

Just as today software relies on hardware to execute the physical tasks required to run programs, tomorrow it will rely on dataware to handle the data storage and management tasks.

Using real-time data collaboration, dataware allows multiple applications and human users to read and write to a shared data repository that supports not only operational workloads, but also data analytics. Changes made in the repository are instantly reflected across an organization’s environment without the need for integration because all applications rely on the same, singular copy of the data.

As you can imagine, a change this fundamental requires a lot of buy-in, but even as the broader goals of dataware gain traction, the technology is immediately useful for solving some of the problems created by the approaches of the past. In this article, I will examine what needs to happen for dataware to become the dominate paradigm as well as what you can already do with dataware today.

Remaining Hurdles

Realizing the full impact of dataware requires applications designed to use dataware in the place of an operational database. For in-house apps, this is relatively simple. Once an organization adopts dataware, its developers can architect any new software they design to use that system. Unfortunately, most companies also rely on a lot of third-party software.

In order for dataware to become the dominant design pattern moving forward, software vendors must create products in a new way. As more of their customers build with dataware, the pressure on these companies will increase because their users will recognize the benefits of applications designed with dataware in mind. An added hurdle is the need for all of these organizations to agree on a standard way for their applications to share data without producing copies of it.

Fortunately, this work is already underway. In Canada, the CIO Strategy Council and the Data Collaboration Alliance are developing a national standard for zero-copy integration. Although only implemented at the national level, the Canadian example will be able to serve as a template for other broader standards in the future. Standards like this one will lower the bar for dataware system interoperability, making it more likely for the approach to catch on.

Adoption Drivers

Even if the grand vision of a world without integration is still some ways off, dataware can provide immediate value for organizations by serving as a data fabric. Dataware can connect to legacy software systems that rely on siloed databases and enable them to collaborate on data even if they were designed before dataware existed. This pattern still requires some integration—a single point of connection between the software and the dataware—but removes the need for any additional future integration. Once an application can communicate with the dataware, it never has to integrate directly with other software in order to utilize their data. Instead of the number of integrations growing exponentially with every new application, the number of integrations grows linearly.

Leveraging dataware as a data fabric creates a central point of access for decentralized data. Analysts can pull the data they need from across any of the company’s source systems using a single interface. Two unique features help dataware stand out from other approaches for creating a data fabric:

1. Model Abstraction. Dataware leverages a shared data model that maps to the data models of the underlying systems. This allows analysts to write abstract queries that don’t break when aspects of the underlying systems change. This out-of-the-box capability also reduces the complexity of the federated queries data fabrics typically rely on.
2. Autonomous Data. Once an organization has connected their source systems in a dataware-facilitated data fabric, the data becomes independent of any one system. As a result, future applications built on top of the dataware require no additional integration at all. They connect to all of the other systems from day one because they leverage data that is shared between all of the those systems.

Conclusion

Dataware has the potential to upend application development as we know it. That level of change will require continued buy-in and wider adoption, but in the meantime dataware still delivers day-to-day value. Using dataware to create data fabrics allows organizations to realize short term benefits while they stay ahead of a potentially larger paradigm shift.

Want to learn more? Check out Joe’s white paper The Rise of Dataware: An Integration-Minimizing Approach to Data Architecture.