How to use IoT Hub with scw CLI

IoT Hub with CLI

In this tutorial we will use the API through the Scaleway command line interface CLI. This will show you how to create Hubs and Devices, as well as more advanced features of the Scaleway Elements IoT Hub: Hub Events and Routes.

You can find the API reference documentation at the following link: IoT API

Requirements: You have:

  • created an account and are logged into console.scaleway.com
  • mosquitto-clients (mqtt client), and jq (json parsing tool) installed on your computer to follow this guide.

CLI setup and configuration

If you already have configured the Scaleway command-line interface, you can skip this section. However, we recommend checking it for updates.

1 . Follow the CLI installation steps.

2 . Configure your CLI by running the following command and answering the questions:

scw init

Setting up the hub

The Hub creation is done through the scw iot hub create command, your CLI configuration already includes the project ID, you can optionally set:

  • A name, with name="my_first_hub". For this tutorial purposes, the name is not important.
  • A product plan, with product-plan="plan_dedicated".

1 . Save the output to a hub.json file to make it easier later, so we need to tell the CLI to output as json:

scw iot hub create -o json > hub.json

jq < hub.json

The file hub.json will contain come content as like the following example:

{
  "id": "b20c3639-9030-496c-a1b2-6feb15846726",
  "name": "cli-hub-cocky-hugle",
  "status": "enabling",
  "product_plan": "plan_shared",
  "enabled": true,
  "device_count": 0,
  "connected_device_count": 0,
  "endpoint": "iot.fr-par.scw.cloud",
  "disable_events": false,
  "events_topic_prefix": "$SCW/events",
  "region": "fr-par",
  "created_at": "2021-04-26T08:46:33.436Z",
  "updated_at": "2021-04-26T08:46:33.436Z",
  "project_id": "<your project ID>",
  "organization_id": "<your organization ID>",
  "enable_device_auto_provisioning": false,
  "has_custom_ca": false
}

2 . Poll the hub status until it is ready:

scw iot hub get $(jq -r '.id' hub.json) | grep Status

At some point, the status will switch to ready.

Set up the devices

Now create 2 devices. You just need to provide:

  • The Hub ID. This is the "id" field from the JSON response received while creating a hub.
  • (Optional) A name. Again, the name is not important for this tutorial.

Note: As this tutorial aims to be simple and straightforward, the following commands are allowing the device to connect using insecure protocols, such as plain text MQTT or MQTTs without mutual authentication. In production, you should Deny Insecure connections to have the highest level of security. This is done by setting the field allow-insecure to false.

1 . Save the response to a file so we can use the fields later.

scw iot device create \
  hub-id=$(jq -r '.id' hub.json) \
  allow-insecure=true \
  -o json > dev1.json

jq < dev1.json

The file dev1.json should contain something similar to:

{
  "device": {
    "id": "0a184d04-aa69-43e5-8fbf-0ee0793aea43",
    "name": "cli-device-pensive-bassi",
    "description": "",
    "status": "enabled",
    "hub_id": "b20c3639-9030-496c-a1b2-6feb15846726",
    "last_activity_at": "1970-01-01T00:00:00Z",
    "is_connected": false,
    "allow_insecure": true,
    "allow_multiple_connections": false,
    "message_filters": {
      "publish": {
        "policy": "reject",
        "topics": []
      },
      "subscribe": {
        "policy": "reject",
        "topics": []
      }
    },
    "created_at": "2021-04-26T09:36:10.708Z",
    "updated_at": "2021-04-26T09:36:10.708Z"
  },
  "certificate": {
    "crt": "<certificate here>",
    "key": "<certificate key here>"
  }
}

2 . Now create a second device:

scw iot device create \
  hub-id=$(jq -r '.id' hub.json) \
  allow-insecure=true \
  -o json > dev2.json

jq < dev2.json

Subscribe and Publish

Now that everything is set up, let’s simulate 2 devices and send data.

1 . Setup the subscriber:

# In one terminal
mosquitto_sub \
  -h $(jq -r '.endpoint' hub.json) \
  -i $(jq -r '.device.id' dev1.json) \
  -t mytopic/mysubtopic

2 . Run the publisher:

# In another terminal
mosquitto_pub \
  -h $(jq -r '.endpoint' hub.json) \
  -i $(jq -r '.device.id' dev2.json) \
  -t mytopic/mysubtopic \
  -m 'Hello, world!'

You should see the subscriber receive the Hello, world! message.

Setting up TLS mutual authentication

If you require security, you can also connect your device to the Hub using TLS mutual authentication. With this method, the Hub can check the device’s identity, and the device can check the Hub’s identity.

Note: It is possible to connect to the Hub using TLS but without Mutual authentication. In this case, the device certificates are not needed as the Hub does not need to check the device identity. But the Hub certificate will still be needed as your client must check the hub’s identity.

1 . Start by downloading the IoT Hub CA:

curl -sS -O https://iot.s3.nl-ams.scw.cloud/certificates/fr-par/iot-hub-ca.pem
sha1sum iot-hub-ca.pem
# 13cf3e59ed52d4c4b6bc249e85539d5fd5d572fb  iot-hub-ca.pem

2 . Then extract the certificates from the device JSON files, so that the mosquitto clients may use them:

jq -r '.certificate.crt' dev1.json > dev1.crt
jq -r '.certificate.key' dev1.json > dev1.key
jq -r '.certificate.crt' dev2.json > dev2.crt
jq -r '.certificate.key' dev2.json > dev2.key

3 . Run the same test as before, but with the added security:

# In one terminal
mosquitto_sub \
  -h $(jq -r '.endpoint' hub.json) -p 8883 \
  --cert dev1.crt --key dev1.key --cafile iot-hub-ca.pem \
  -i $(jq -r '.device.id' dev1.json) \
  -t mytopic/mysubtopic
# In another terminal
mosquitto_pub \
  -h $(jq -r '.endpoint' hub.json) -p 8883 \
  --cert dev2.crt --key dev2.key --cafile iot-hub-ca.pem \
  -i $(jq -r '.device.id' dev2.json) \
  -t mytopic/mysubtopic \
  -m 'Hello, SECURE world!'

Note: You can mix MQTT and MQTTs clients on the same hub.

You can harness the real power of MQTT Pub/Sub with a few more lines of reading.

IoT Hub Events

Device information, such as events or errors, are reported on your Hub under the default topic prefix $SCW/events. For more information on Hub Events, you can read the IoT Hub events documentation.

In the following example we will trigger an event by trying to connect without security on a device where security is required.

1 . Connect to previously created dev1 device as events consumer (we want only error events)

# In one terminal
mosquitto_sub \
  -h $(jq -r '.endpoint' hub.json) -p 1883 \
  -i $(jq -r '.device.id' dev1.json) \
  -t '$SCW/events/error/#'

2 . Then create a secured-only device and try to connect it in insecure mode:

scw iot device create \
  hub-id=$(jq -r '.id' hub.json) \
  allow-insecure=false \
  -o json > secure-dev.json

jq < secure-dev.json
# In another terminal
mosquitto_pub \
  -h $(jq -r '.endpoint' hub.json) -p 1883 \
  -i $(jq -r '.device.id' secure-dev.json) \
  -t mytopic/mysubtopic \
  -m 'This should fail'

The device dev1 will receive the following message, on the topic $SCW/events/device/<secure-dev-id>/error:

{
  "time": "2021-04-26T09:48:03Z",
  "severity": "error",
  "object-type": "device",
  "object-id": "dev3-id",
  "msg": "connection refused because this device requires mutual TLS authentication",
  "packet": "CONNECT: dup: false qos: 0 retain: false rLength: 48"
}

Routes

Routes are integrations with the Scaleway ecosystem: they can forward MQTT messages to Scaleway services.

The documentation is available at the following link: IoT Hub Routes

S3 Route

The S3 route allows you to put the payload of MQTT messages directly into Scaleway’s Object Storage.

Important: This section is an addition to the “Getting started with IoT Hub” above, make sure to follow it first!

As a prerequisite, you should have s3cmd installed and configured for Scaleway. See this guide if you don’t have it installed yet.

1 . Run the following commands in a terminal on your computer:

BUCKET="my-bucket-$RANDOM" # Buckets are globally unique, suffix with a random number
REGION="fr-par"
PREFIX="iot/messages"
# Create the bucket
s3cmd mb --region "$REGION" "s3://$BUCKET"
# Grant write access to IoT Hub S3 Route Service to your bucket
s3cmd setacl --region "$REGION" "s3://$BUCKET" --acl-grant=write:555c69c3-87d0-4bf8-80f1-99a2f757d031:555c69c3-87d0-4bf8-80f1-99a2f757d031
# Create the IoT Hub S3 Route
scw iot route create \
  hub-id=$(jq -r '.id' hub.json) \
  name=route-s3-cli topic="hello/world" \
  s3-config.bucket-region="$REGION" \
  s3-config.bucket-name="$BUCKET" \
  s3-config.object-prefix="$PREFIX" \
  s3-config.strategy=per_topic

The output will contain something like:

ID                     5ce53577-6905-4b22-970f-d1e345e7345a
Name                   route-s3-cli
HubID                  b20c3639-9030-496c-a1b2-6feb15846726
Topic                  hello/world
Type                   s3
CreatedAt              now
S3Config.BucketRegion  fr-par
S3Config.BucketName    my-bucket-26793
S3Config.ObjectPrefix  iot/messages
S3Config.Strategy      per_topic
UpdatedAt              now

2 . Publish and see the result.

sleep 5 # wait a little for the route to start
mosquitto_pub \
  -h $(jq -r '.endpoint' hub.json) \
  -i $(jq -r '.device.id' dev2.json) \
  -t hello/world \
  -m 'Hello, world!'

3 . An object iot/messages/hello/world should now be stored in your bucket with “hello world” as contents. Retrieve it using s3cmd.

4 . Retrieve the object using s3cmd:

s3cmd get --region "$REGION" "s3://$BUCKET/$PREFIX/hello/world"
cat world

Database Route

Database route allows you to store messages in your database.

When creating such a route for one of you hubs, you need to specify a topic (wildcards are allowed), a database (with valid credentials) and a query to execute. That query may contains $TOPIC and $PAYLOAD variables.

The route will subscribe on this hub to this topic, and execute the query onto the given database for each received message:

  • First, $TOPIC and $PAYLOAD are replaced with the topic and payload of the received MQTT message
  • Then the generated query is executed

NOTE: In PostgreSQL, the topic database field must be a of text type, and payload a bytea.

This tutorial covers the PostgreSQL database system. You can use a Scaleway Database instance, or any other PostgreSQL instance publicly accessible.

Let's practice

Important: This section is an addition to the “Getting started with IoT Hub” above, make sure you follow it first!

As a prerequisite, you must have a working PostgreSQL database, with valid credentials (username and password).

1 . Run the following commands in a terminal on your computer:

# Database settings
DBENGINE=postgresql
DBHOST=<your db host>
DBPORT=<your db port>
DBNAME=<your db name>
DBUSER=<your db user>
DBPASS=<your db password>

# Create the target database table
psql -h $DBHOST --port $DBPORT -U $DBUSER -d $DBNAME -c '
  CREATE TABLE messages (
    time timestamp,
    topic text,
    payload bytea
  )
'

# Create the IoT Hub Database Route
# The query will insert message topic and payload with current timestamp
scw iot route create \
  hub-id=$(jq -r '.id' hub.json) \
  name=route-db-cli \
  topic="hello/world" \
  db-config.engine="$DBENGINE" \
  db-config.host="$DBHOST" \
  db-config.port=$DBPORT \
  db-config.dbname="$DBNAME" \
  db-config.username="$DBUSER" \
  db-config.password="$DBPASS" \
  db-config.query='INSERT INTO messages VALUES (NOW(), $TOPIC, $PAYLOAD)'

The output will contain something like

ID                 2251e2b1-c616-4a7e-9e72-b658da656424
Name               route-db-cli
HubID              b20c3639-9030-496c-a1b2-6feb15846726
Topic              hello/world
Type               database
CreatedAt          now
DbConfig.Engine    postgresql
DbConfig.Host      127.0.0.1
DbConfig.Port      5432
DbConfig.Dbname    route_tests
DbConfig.Username  jdoe
DbConfig.Password  <your_pass>
DbConfig.Query     INSERT INTO messages VALUES (NOW(), $TOPIC, $PAYLOAD)
UpdatedAt          now

2 . Publish a message and check whether it is inserted into the message table.

sleep 5 # wait a little for the route to start
mosquitto_pub \
  -h $(jq -r '.endpoint' hub.json) \
  -i $(jq -r '.device.id' dev2.json) \
  -t hello/world \
  -m 'Hello, world!'
psql -h $DBHOST --port $DBPORT -U $DBUSER -d $DBNAME -c "SELECT * FROM messages"

More examples including MySQL and more advanced features are available on the database route tips & tricks page.

Rest Route

Important: This section is an addition to the “Getting started with IoT Hub” above, make sure you follow it first!

Rest route allows you call any http(s) endpoint on received MQTT message. You can choose the HTTP verb use to call your REST uri, as well as adding extra headers.

We can see what a rest route would publish on a rest API by simply listening to the port 80 on a public IP.

You can use a Scaleway instance, or any other machine with a public IP address.

1 . Launch the following command on the machine, as root:

nc -p 80 -l

2 . Define a variable with the public IP address of your server

RESTHOST=<the_public_ip_address>

3 . Create the route by running the following command:

# Create the IoT Hub Rest Route
scw iot route create \
  hub-id=$(jq -r '.id' hub.json) \
  name=route-rest-cli \
  topic="hello/world" \
  rest-config.verb=post \
  rest-config.uri="http://$RESTHOST/" \
  rest-config.headers.X-My-Header="Tutorial"

4 . Publish a message and checks it triggers a request on the server:

sleep 5 # wait a little for the route to start
mosquitto_pub \
  -h $(jq -r '.endpoint' hub.json) \
  -i $(jq -r '.device.id' dev2.json) \
  -t hello/world \
  -m 'Hello, world!'

5 . On your server the above nc output should be:

POST / HTTP/1.1
Host: <the_public_ip_address>
User-Agent: Go-http-client/1.1
Content-Length: 13
X-Mqtt-Retain: false
X-Mqtt-Topic: hello/world
X-My-Header: Tutorial
Accept-Encoding: gzip

Hello, world!

Learn more about Scaleway IoT Hub, discover how to add Devices to the hub, check the IoT Hub metrics or get started in a few clicks with the IoT Hub Kickstarts.

Discover the Cloud That Makes Sense