Data Warehouse for ClickHouse® - Concepts

AutoscalingLink to this anchor

Autoscaling refers to the ability of a Data Warehouse for ClickHouse® deployment to automatically adjust the number of instances without manual intervention. Scaling mechanisms ensure that resources are provisioned dynamically to handle incoming requests efficiently while minimizing idle capacity and cost.

Bottomless storageLink to this anchor

Bottomless storage is a feature that allows ClickHouse® to separate compute and storage by offloading data to external object storage, such as Amazon S3-compatible services, while keeping frequently accessed data cached locally for fast queries. In ClickHouse®, this is implemented by transparently moving older or less-used data to remote storage, enabling virtually unlimited storage capacity without sacrificing performance for active workloads.

Refer to the official ClickHouse® documentation for more information.

ClickHouse®Link to this anchor

ClickHouse® is a high-performance, column-oriented, distributed database management system designed for real-time analytics. It is optimized for handling large volumes of data with fast query performance, making it ideal for applications requiring up-to-date insights. ClickHouse® stores data in a columnar format, which reduces I/O operations and speeds up query execution. It supports distributed processing across multiple nodes, enabling horizontal scaling and fault tolerance through replication. ClickHouse® provides a powerful SQL interface and offers advanced features like real-time data ingestion, compression, and indexing, making it a robust solution for analytical workloads.

ClickHouse® HTTP consoleLink to this anchor

The ClickHouse® HTTP console is a web interface that allows you to interact easily with your Data Warehouse deployment from the Scaleway console by executing SQL queries.

Note

Make sure to enter valid credentials in the top-right fields before executing queries.

Column-oriented storageLink to this anchor

ClickHouse® stores data in a column-oriented format, which significantly optimizes read performance for analytical queries. By storing data in columns rather than rows, ClickHouse® reduces the number of I/O operations needed during query execution, as it only reads the necessary columns from disk.

CompressionLink to this anchor

ClickHouse® uses advanced compression algorithms to reduce storage requirements and improve query performance by minimizing data transfer. Compression not only helps in saving disk space but also accelerates data retrieval and processing by reducing the amount of data that needs to be read from storage and transferred over the network.

Distributed processingLink to this anchor

ClickHouse® supports distributed processing across multiple nodes, allowing it to handle extremely large datasets efficiently and scale horizontally. This architecture enables ClickHouse® to distribute data and queries across a cluster, improving performance and reliability by leveraging the combined resources of all nodes.

Horizontal scalingLink to this anchor

Horizontal scaling refers to the process of adding more nodes to the cluster to increase its capacity and performance. This approach allows the cluster to handle larger datasets and higher query loads by distributing the data and processing tasks across additional nodes. Data Warehouse for ClickHouse® deployments scale automatically according to the incoming workload.

IndexingLink to this anchor

ClickHouse® employs various indexing techniques, such as primary key and skip indexes, to speed up query execution and data retrieval. The primary key index allows for efficient point lookups and range queries, while skip indexes help in quickly skipping over large chunks of data that do not match query conditions, thus reducing the overall query time.

NodeLink to this anchor

In the context of a distributed Data Warehouse for ClickHouse® cluster (also called “deployment”), a node refers to an individual instance that stores and processes a portion of the data. Each node participates in data distribution, query execution, and replication to ensure balanced load, fault tolerance, and high availability. Nodes communicate with each other to coordinate tasks, execute queries in parallel, and maintain synchronized data replicas. They are configured with specific settings to define their roles and manage resources, allowing the cluster to scale and perform efficiently.

Replica setLink to this anchor

A replica set consists of multiple nodes that store identical copies of the same data. This setup ensures fault tolerance and high availability by providing redundancy. If one node in the replica set fails, another node can take over, ensuring continuous data access and processing. ClickHouse® automatically handles data replication and failover, making it a reliable solution for mission-critical applications.

SQL supportLink to this anchor

ClickHouse® provides a powerful SQL interface, enabling users to perform complex queries and data manipulations using familiar SQL syntax. This extensive SQL support includes a wide range of functions and features, such as subqueries, window functions, and user-defined functions, making it accessible to both analysts and developers.

Vertical scalingLink to this anchor

Vertical scaling refers to the process of increasing the resources of individual nodes within the cluster. Vertical scaling enhances the performance and capacity of individual nodes, allowing them to handle larger datasets and more complex queries more efficiently. Vertical scaling is often used in conjunction with horizontal scaling to optimize performance and resource utilization in a Data Warehouse for ClickHouse® deployment.