What is data fabric?
How does it affect your data estate?
A data fabric is an information architecture that unifies data across an organization. According to Techopedia, data fabric describes “a distributed IT architecture in which data is governed the same way whether it is located on premises, in the cloud, or at the edge of a network”.1
Rather than being a single piece of technology, data fabric is a holistic data and AI strategy that helps organizations leverage all existing and future investments within their data estate. The purpose of a unified data fabric is to ensure that an organization's data is always accessible to all authorized parties, regardless of where it is stored.
In the Gartner article Data Fabric Architecture is Key to Modernizing Data Management and Integration, author Ashutosh Gupta notes that data fabric “leverages both human and machine capabilities to access data in place or support its consolidation where appropriate [and it] continuously identifies and connects data from disparate applications to discover unique, business-relevant relationships between the available data points”.2
Data Fabric for Dummies author Ed Tittel states that “data fabric is a living, always-evolving collection of capabilities that grows and changes along with whatever organization it serves”.3
One of today’s biggest business challenges is figuring out how to close the critical gap between the available data and information, and transforming as much of that data and info into resources like knowledge and insights. These insights are necessary for creating personalized, compelling customer experiences, and cutting edge products and services that enhance a business’ operational efficiency.
As technology becomes more scalable and people are more connected, the complexity and gaps in leveraging actionable insights increase. According to Forrester, “an average of between 60% and 73% of all data within an enterprise goes unused for analytics”.4 Today’s businesses struggle to close that gap.
Data Fabric as Modern Data Architecture author Alisa LaPlante notes that the explosion in the growth of data and continued fragmentation has almost rendered data centralization a distant dream. LaPlante points out that “with data being generated and stored everywhere from the data center to the edge to the cloud, having a single centralized [system of] record doesn’t work anymore”.5
According to LaPlante, “data fabric helps businesses virtualize their data, instead of relocating it to a centralised location. Security controls and authentication measures can be applied as though they were all in one place. It also makes things much easier for the company's users. Virtualizing, not centralising, is the foundation for a strong data fabric”.5
LaPlante argues that data fabric offers the following benefits:
Data fabric is a gateway for businesses looking to move beyond older data management practices and optimize their resources. According to a StrategyR market report, “the market for software products and services that facilitate the creation and management of data fabrics will grow to be $3.7 billion annually by 2026”.5
Data fabric is often viewed as a gradual offspring of earlier legacy systems with hardware and a network but no overarching network management system for getting data where it needed to go.
Forrester finds that a data fabric “minimizes the complexity [of a data estate] by automating processes, workflows, and pipelines, generating code and streamlining data to accelerate various use cases”.6
In Data Fabric for Dummies, author Ed Tittel declares that “data fabric brings worthwhile change and numerous benefits to those who buy into the vision”.3 Here are several examples of data fabric's benefits:
Hybrid Cloud & Data Fabric for Dummies authors Larry Freeman and Lawrence C. Miller suggests that data fabric is an infrastructure “built for today, but designed for tomorrow”.9 According to Freeman and Miller, “as cloud popularity grows, [it is likely that] more and more corporate data will move into a shared cloud environment, with only the most sensitive data staying within the confines of the data center. A data fabric is designed to enable [and accelerate] this shift to the cloud”.9
While the benefits of data fabric are multifold, there are many blockers that prevent organizations from successfully adopting a solid data fabric architecture. Challenges to data fabric adoption include, but aren’t limited to, the following:
According to LaPlante, “many companies spend a lot of time, effort, and money [attempting] to figure out if they can access their data, as well as define and add value to it. However, they don't always use analytics to acquire insights. And almost all of them fail to apply what they've learned and [then optimize to] shut the loop. When data fabrics become commodities, how well organizations execute in a closed-loop analytics system will determine how they compete in the future”.5
Organizations today often own and use their data assets. A data fabric helps to unify all these assets, creating a “consolidated data management environment that extends across an organization’s edge-to-core-to-cloud infrastructure for all platforms and applications”.3
Tittel argues that data fabric implementation usually requires a modernization process of “[taking] existing fragmented and siloed approaches to managing, storing and situating data into the data fabric’s single, consistent, policy-driven, self-service environment, supported by DevOps and DataOps principles”.3
Tittel states that “modernization is a vital job for organizations seeking to deploy a data fabric [and that it] involves nothing less than a review of all data assets relevant to a business case, and all applications and services that consume and produce data [while trying to open] the data fabric’s umbrella to cover everything”.3
Data fabric, like so many other technology-based elements in business and the world today, has a human factor. Tittel notes that “applications come with stakeholders, users, and developers, all of whom must understand and buy into that fabric [and enterprises] risk fighting “shadow IT” and other end-arounds that favor “quick and dirty” over consistent and coherent”.3
Tittel notes that following the financial crises and the pandemic, there’s increased “demand that any technology investments provide a faster [ROI which] puts added pressure on existing investments [and] it encourages cautious and critical consideration of new [investments]”.3 He argues that the market conditions “[benefit] use cases at the high end of the volume and variety curve where bigger payoffs prevail”.3
Tittel notes that for data fabric adoption, modern data fabrics requires the enterprise to leverage DataOps and must use the “CI/CD [continuous integration, continuous delivery] Agile methodology [that] requires data analysts, business stakeholders, and IT personnel to stop working in disparate environments [and] collaborate to ensure [data comprehension and] requires data analysts, business stakeholders, and IT personnel to stop working in disparate environments”.3
Tittel notes that this integration will produce “self-monitoring and self-measurement for the data fabric so that it keeps getting better at managing data through its entire lifecycle”.3 By this logic, the data fabric is in a persistent state of optimization, and is constantly performing better than it had been previously.
Data fabric helps you make better decisions through improved compute performance across data channels. Ashutosh Gupta argues that “to deliver business value through data fabric design, D&A leaders should ensure a solid technology base, identify the required core capabilities, and evaluate the existing data management tools”.2
Gupta notes that “Contextual information lays the foundation of a dynamic data fabric design. There should be a mechanism (like a well-connected pool of metadata) that enables data fabric to identify, connect, and analyze all kinds of metadata such as technical, business, operational, and social”.2
Gupta argues that for data sharing to work best, enterprises must activate their metadata. For this metadata to become active, their data fabric must do the following:
Gupta notes that the reason data fabric needs to make and curate knowledge graphs, is because these graphs “[enrich] data with semantics [and help] data and analytics leaders to derive business value”.2 He states that the “semantic layer of the knowledge graph makes it more [intuitive and] the analysis easy for D&A leaders”2. Lastly, the semantic layer “adds depth and meaning to the data usage and content graph, allowing AI/ML algorithms to use the information for analytics and other operational use cases”.2
One of the benefits of data fabric adoption is how flexible it is in terms of data delivery. It easily works with multiple data delivery methods, which help its enterprise to accommodate a vast array of data consumers, extending from traditional IT through finance and business.