The central component of the unified data platform is a cloud data warehouse. A data warehouse is a database dedicated to storing and analyzing data to reveal trends, patterns, and correlations that lead to valuable insights and information. A data warehouse is built for BI, can be easily connected to most BI tools, and can integrate with modern programming languages.
Data warehouses also have features that make them different than traditional on-line transaction processing (OLTP) databases. OLTP databases usually store their data in a row-based format. However, data warehouses store data in a columnar based storage format. Columnar storage enhances query speed, memory performance, and I/O speed.
Data warehouses are also designed to store vastly more data than a traditional database at a fraction of OLTP databases’ cost. Data is also compressed, thus enhancing storage capacity and cost-efficiency.
Finally, data warehouses use a distributed cluster architecture across multiple nodes similar to the high-performance distributed computing paradigm. A cluster architecture greatly enhances query speed, durability, and availability.
This post is part of a 23 part mini-series about implementing a cost-effective modern data infrastructure for a small organization. This is a small part of a whitepaper that will be released at the end of this series.