DATA WAREHOUSE HEALTH CHECK

A data warehouse is a bit like a sports car engine: when well maintained and serviced, it delivers unparalleled value to its users, but when neglected it becomes very expensive to return to proper and sustainable performance.

As anyone who has worked with a Data Warehouse can affirm, the DWH is a complex and often fragile ecosystem of technologies, data inputs and outputs, and governance processes. In the more than 30 years since Bill Inmon first proposed the Corporate Information Factory, a deep understanding and comprehensive set of disciplines and best practices have emerged for the development and ongoing maintenance of the data warehouse. Unfortunately, these best practices are not always followed, and it can be necessary from time to time to assess the health of a data warehouse against the business requirements (which are constantly evolving) and best practices, which make the data warehouse sustainable and in the long run more economical.

The DWH Health Check will give you a prioritised, costed and detailed set of initiatives that can be used to support a business case for addressing the missing business benefits and capabilities that the DWH should be capable of supporting.

The Data Warehouse Health Check evaluates the following dimensions of performance:

Business Requirements Assessment

  • What are the expected business benefits that the DWH is needed to provide?

  • How have these requirements changed since the DWH was designed/commissioned?

  • How well is the DWH meeting these needs now, and how significant is the delta?

  • What expectations from the business, for information subject areas or for capabilities (such as near real time analytics or predictive modelling) are not being met and what is needed to address the shortfall?

Information Architecture Assessment

  • Are there significant departures from best practice and if so do they create sustainability, security or business value issues?

  • Has there been an evolution in Business Requirements which means that areas of the DWH are no longer needed, or are needed but are no longer fit for purpose and require remediation?

  • Are there gaps in metadata management, data lineage, the data dictionary, or other key knowledge assets which create risk to the business?

  • Is sensitive data such as PII appropriately identified and protected?

  • Are there data quality issues (real or perceived) and trust issues which impact the value that can be derived from the DWH investment?

  • Are there areas which fall short of legislative, contractual or ethical constraints on how data is used, or where there is a risk of inappropriate data access or usage?

  • Is there a backlog of enhancements or projects that are unable to be delivered in a timely manner due to the complexity and/or cost of the DWH, and if so can alternative ways of meeting these business needs be created, or is it more cost effective to remediate the issues in the DWH?

Technical Architecture Assessment

  • The end-to-end technical environment is assessed against the capability needs of the business, and any gaps identified.

  • Are there departures from best practice or sustainability that have created a technical debt, and if so what is the most cost effective way forward?

  • Are novel approaches to the architecture needed, such as Operational Data Stores, Data Lakes, real-time/streaming analytics, or semi- or unstructured data repositories, and if so how can these be created to leverage and complement the existing investment?

  • Are there legacy technologies that should be “sunsetted”; shelfware; or technology gaps which create risk, usability, efficiency or sustainability issues?

Organisational Assessment

  • Have roles and responsibilities for the data warehouse been identified and documented? 

  • Has ownership of the data warehouse been addressed from the perspective of strategic business objectives and direction setting? From the perspective of tactical enhancement and ongoing business needs? From perspectives of information management and technical support and responsibilities? 

  • Has the key issue of business/IT collaboration been articulated and addressed? 

  • Has the need for ongoing operational support and tuning, in parallel with continuing development, been considered? 

  • Have data stewardship roles, data quality practices, and metadata management responsibilities been implemented? Do both business and IT organizations have appropriate responsibilities here? 

  • Has the data warehouse been positioned organizationally within a broader context of enterprise information and knowledge management? 

  • Has a plan, process, and structure been established for ongoing training of users and enhancement of technical skills? 

  • Has any structure been put in place for ongoing monitoring of data warehouse quality, and for periodic assessments, as needed?