How to import data into a Data Warehouse for ClickHouse® deployment

Data federation allows you to query and combine data from multiple sources, enabling seamless analytics across different databases or storage systems without the need to move or duplicate data.

Scaleway Data Warehouse for ClickHouse® allows you to quickly import any type of compatible data into bottomless storage with minimal configuration.

Before you startLink to this anchor

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• Signed up to the private beta and received a confirmation email.
• Created a Data Warehouse deployment

How to import data using the ClickHouse® CLILink to this anchor

Connecting to the ClickHouse® CLILink to this anchor

1. Click ClickHouse® under Data & Analytics on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the Data Warehouse deployment you want to connect to. The overview tab of the deployment displays.

3. Click Connect next to Frameworks in the Connect to your deployment section. The connection wizard displays.

4. Select Protocols, then copy the command in the ClickHouse® CLI tab.

5. In a terminal, paste and execute the copied command to connect to your deployment. Make sure to replace the placeholders with the corresponding values.

   clickhouse client \
   --host <YOUR_DEPLOYMENT_URL> \
   --port 9440 \
   --secure \
   --user scwadmin \
   --password '<YOUR_DEPLOYMENT_PASSWORD>'
   

6. You are now connected to your deployments using the ClickHouse® CLI, you can now import data.

Importing data into your deployment from an Object Storage bucketLink to this anchor

There are several ways to import data into your Data Warehouse for ClickHouse® deployment, depending on how your data is stored. One of the most common ways is to import it from an Object Storage bucket in any format supported by ClickHouse®.

• s3 table function
• s3 storage engine
• ClickHouse® Client

INSERT INTO your_table
SELECT *
FROM s3('https://my-bucket.s3.scaleway.com/data/my_data.csv', 'CSV', 'column1 String, column2 Int32')

    CREATE TABLE your_table_s3
    ENGINE = S3('https://my-bucket.s3.scaleway.com/data/my_data.csv', 'CSV', 'column1 String, column2 Int32')

clickhouse-client --query="INSERT INTO your_table FORMAT CSVWithNames" \
--url "https://my-bucket.s3.scaleway.com/data/my_data.csv" \
--input_format_with_names=1