The data warehouse (DW) has proven to be the main ingredient for business intelligence (BI) and advanced analytics. It’s very success, though, has led to a situation where companies are constantly struggling to meet Service Level Agreements (SLA), scale and adapt to additional, large internal and external data source. Just running a DW on-premises has become a costly and requires constant maintenance and monitoring, despite the business advantages of having an integrated view of data across the enterprise. The cloud data warehouse emerges as an attractive alternative. The cloud data warehouse is easier to manage, scale and modify. It allows the organization to focus on data and business value instead of maintenance and performance. However, not all cloud data warehouses are equal.
What is a Cloud Data Warehouse?
What is a cloud data warehouse? To understand how a cloud data warehouse works, it’s first necessary to grasp the basics of data warehouses in general. A data warehouse is a repository of integrated, historical business data. The core value is derived from a data warehouse when data analysis can be performed without performance degradation and in meeting SLAs.
By having an integrated, historical view of data across the enterprise’s transactions, the data warehouse supports adhoc analysis, BI, extracts, analytics and other needs.
Until about five years ago, DWs were hosted in on-premises systems. Like any large database, they had their own compute and storage infrastructure. Infrastructure admins looked after the hardware and network supporting the DW. Database administrators took care of the inner workings of the DW, tuning and adjust as new demands were put on the DW while trying to make sure existing workloads were not impacted. This arrangement worked and provided immense value to the business, but today there are better options.
A cloud data warehouse puts all the infrastructure of the data warehouse into a public cloud hosting environment. The database software and analytics tools run in the cloud. The database servers, network and storage volumes are all virtualized and available when needed.
Benefits of the Cloud Data Warehouse
The cloud data warehouse, offered as Software-as-a-Service (SaaS), offers several advantages over the on-premises version. It removes the need for costly infrastructure, support and maintenance that come along with that model. Teams are not required to managing and maintaining the hardware and software, tuning or general administration. With a cloud data warehouse, teams can more focus on the data and business value.
An on-premises warehouse requires capital expense (CapEx) and most of its upfront investment for the hardware and software. Then, afterwards, the DW needs OpEx to pay employees to maintain the data warehouse. A cloud-based warehouse is about one third to one half the cost of on premises warehouses. The cost is incurred “upon use” of the cloud warehouse and are considered operating expense (OpEx).
Providers of cloud data warehouses shoulder the CapEx burden themselves and rent out the assets to their customers. It’s similar to a real estate developer who builds a building and rents out apartments to tenants. OpEx for a DW benefits companies to conserve cash and reduce upfront investment.
Elasticity is also a benefit of the cloud data warehouse. Available capacity is “limitless”, as far as the customer is concerned. It can be scaled, or scaled down, as needed and when needed. If the customer needs 10 terabytes of data warehouse storage and additional compute capacity in January for a big analytical project, but only needs 3 terabytes on an ongoing basis, this can be done in the cloud. If that same customer decide they need 100 terabytes and compute to match in July, for some surprise reason (like litigation or compliance), they can also put that capacity online when needed.
In contrast, getting even a fraction of this capacity on premises will be months of effort. Any capacity added, generally cannot be “removed” as it is “purchased and owned” in most cases.
Working with the Right Cloud Data Warehouse
The cloud data warehouse, in general, provides advantages to businesses that embrace the cloud model. However, not every cloud data warehouse is equally advantageous. For example, simply lifting and shifting an existing data warehouse into the cloud may not fit the “data architecture” supported by some of the cloud data warehouse software.
This may require a larger spend to migrate and more expenditure for meeting SLAs. For example, moving a data warehouse model to the cloud depending on the software may require the same effort to tune, monitor and maintain as on premises systems. Most cloud data warehouse software have their roots in on premises systems. Others were on premises systems “migrated” to the cloud. In either case, they don’t take advantage of cloud architecture and innovations.
Snowflake addresses these challenges. Snowflake was built specifically to be hosted in the cloud and take advantage of technology and solution only available in the cloud. The Snowflake cloud data warehouse architecture features centralized, scale-out storage. To this, Snowflake adds multiple, independent compute clusters. It’s a multi-cluster, shared storage model. With this cloud-native architecture, multiple compute can execute independent workloads simultaneously, making efficient use of cloud resources, managing costs, and not affecting performance.
While others may be able to provide separation of compute and storage, Snowflake allows for separation of compute over “shared” storage, a key advantage. This allows for separate, dedicated, “right” sized compute for each of the functions (ex. ETL, adhoc analysis, BI) a data warehouse needs to support. It is also true that using cloud storage provides that “limitless”, burstable or permanent capacity that is needed.
Clarity Insights is one of Snowflake’s top partners. The partnership with Snowflake spans multiple industries, including manufacturing, financial services, media and retail. Working together, Clarity leverages the Snowflake cloud data warehouse to support clients’ cloud-based modernization efforts.
Written by Ali Sajanlal
Snowflake CoE Lead