Snowflake Architecture – Important 3 Layers

Snowflake Architecture is basically combination of shared disk and shared nothing architecture. Similar to Shared disk, snowflake has a persistent shared storage layer and just like shared nothing architecture it makes use of MPP’s for processing of data.

Snowflake architecture is very Unique and consists of three key layers:

• Database Storage
• Query Processing
• Cloud Services

Database Storage

Snowflake restructures data into its own efficient, compressed, columnar format once it is loaded into snowflake. This streamlined data is kept in cloud storage by Snowflake.

All parts of this data’s storage are managed by Snowflake, which also handles metadata, statistics, compression, file size, categorization, and other aspects of data management. Customers cannot directly view or access the data objects that Snowflake stores; instead, they can only access them through SQL query operations that are performed with Snowflake.

Query Processing

The processing layer is where query execution takes place. Snowflake employs “virtual warehouses” to process query requests. Every virtual warehouse comprises of many compute nodes that are allocated by Snowflake from a cloud provider, forming an MPP compute cluster.

Virtual warehouses do not share compute resources with one another; instead, they operate as separate compute clusters. Each virtual warehouse is therefore independent of the other virtual warehouses’ functioning.

Cloud Services

A group of services that coordinate operations within Snowflake make up the cloud services layer. These services connect all of Snowflake’s various parts to handle user requests, from login to query dispatch. Compute instances that Snowflake procures from the cloud provider are likewise utilized by the cloud services layer.

Below Services are managed in this layer:

• Authentication: This service takes care of verifying the users’ identities who access the system and ensures security.
• Infrastructure management: This service is about overall management and maintenance of the hardware and software systems that support the snowflake environment.
• Metadata management: Managing the data that describes other data in the system is known as metadata management, and it is essential to effectively organize and retrieve data.
• Query parsing and optimization: This service takes care of interpretation of user queries and transforms into an efficient execution plan to interact with the database.
• Access control: As the name suggests this defines and regulates who can access what data in the database. and to decide what a user can do with data while maintaining data security and privacy.

Snowflake Architecture – 3 Layers (Image source: Snowflake website)

Summary

• Snowflake architecture comprises below mentioned 3 layers:
  • Database Storage:
    • Restructures data into Snowflake’s format
    • Stored in cloud storage managed by Snowflake
  • Query Processing:
    • Virtual warehouses process queries
    • Form MPP compute clusters
    • Each warehouse operates independently
  • Cloud Services:
    • Coordinate operations within Snowflake
    • Manage authentication, infrastructure, metadata, query parsing, optimization, and access control

More information : Key Concepts & Architecture | Snowflake Documentation