In this tutorial we will use the API through the well known utility curl. This will show you how to create Hubs and Devices, as well as the more advanced features of the Scaleway Elements IoT Hub: Hub Events and Routes.
The API reference is here: IoT API
You will need to have curl
, mosquitto-clients
(mqtt client), and jq
(json parsing tool) installed to follow this guide.
You will need to find 2 things to get you started: a token
and your organization ID
. You can get/create them from your Scaleway console, under the Account > Credentials
page.
Some documentation here: How to generate an API token
Let’s save this for later:
IOT_API="https://api.scaleway.com/iot/v1beta1/regions/fr-par"
SCW_TOKEN="<your scaleway token secret here>"
SCW_ORG="<your organization ID here>"
Let’s learn how to spawn your first IoT platform as a service with Scaleway.
This is as simple as an API call, provide your organization id, a name and a product plan. We will save the output to hub.json
file to make it easier later:
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"organization_id": "'$SCW_ORG'",
"name": "my_first_hub",
"product_plan": "plan_dedicated"
}' $IOT_API/hubs > hub.json
jq < hub.json
hub.json
will contain something like:
{
"id": "a15ddd5b-ff73-4fb3-b043-1c9176dae295",
"name": "my_first_hub",
"organization_id": "<your organization ID>",
"status": "creating",
"product_plan": "plan_dedicated",
"endpoint": "iot.fr-par.scw.cloud",
"created_at": "2018-12-13T15:15:27.057005Z",
"enabled": true,
"device_count": 0,
"connected_device_count": 0,
"events_enabled": true,
"events_topic_prefix": "$SCW/events"
}
We can poll the hub status until it is ready:
curl -sS -H "X-Auth-Token: $SCW_TOKEN" $IOT_API/hubs/$(jq -r '.id' hub.json) | jq -r '.status'
At some point, the status will switch to ready
.
Now let’s create 2 devices, you just have to provide your organization id, hub id and a name. We will save the response to a file so we can use fields later.
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"hub_id": "'$(jq -r '.id' hub.json)'",
"name": "my_first_device",
"allow_insecure": true
}' $IOT_API/devices > dev1.json
jq < dev1.json
dev1.json
should contain something like:
{
"device": {
"id": "ca973002-b87d-457e-8b9b-f5e19796e2b9",
"organization_id": "<your organization ID>",
"name": "my_first_device",
"status": "enabled",
"hub_id": "a15ddd5b-ff73-4fb3-b043-1c9176dae295",
"created_at": "2018-12-14T13:59:43.277945Z",
"allow_insecure": true,
"last_activity_at": "1970-01-01T00:00:00Z",
"is_connected": false,
"message_filters": {
"publish": {
"policy": "reject",
"topics": []
},
"subscribe": {
"policy": "reject",
"topics": []
}
}
},
"crt": "<certificate here>",
"key": "<certificate key here>"
}
Let’s now create a second device:
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"hub_id": "'$(jq -r '.id' hub.json)'",
"name": "my_second_device",
"allow_insecure": true
}' $IOT_API/devices > dev2.json
jq < dev2.json
Ok, we have all set up, let’s simulate 2 devices and send data now!
Setup the subscriber:
# In one terminal
mosquitto_sub \
-h $(jq -r '.endpoint' hub.json) \
-i $(jq -r '.device.id' dev1.json) \
-t mytopic/mysubtopic
Run the publisher:
# In another terminal
mosquitto_pub \
-h $(jq -r '.endpoint' hub.json) \
-i $(jq -r '.device.id' dev2.json) \
-t mytopic/mysubtopic \
-m 'Hello, world!'
The subscriber is receiving the Hello, world!
message. Yay!
If you require security, you can also send your messages with TLS mutual authentication.
You will need the IoT Hub endpoint certificate authority to verify its identity. Let’s download it:
curl -sS -O https://iot.s3.nl-ams.scw.cloud/certificates/fr-par/iot-hub-ca.pem
sha1sum iot-hub-ca.pem
# 13cf3e59ed52d4c4b6bc249e85539d5fd5d572fb iot-hub-ca.pem
Now let’s extract the certificates from the device JSON files, so that the mosquitto clients may use them.
jq -r '.crt' dev1.json > dev1.crt
jq -r '.key' dev1.json > dev1.key
jq -r '.crt' dev2.json > dev2.crt
jq -r '.key' dev2.json > dev2.key
We can now run the same test as previously, with added security:
# In one terminal
mosquitto_sub \
-h $(jq -r '.endpoint' hub.json) -p 8883 \
--cert dev1.crt --key dev1.key --cafile iot-hub-ca.pem \
-i $(jq -r '.device.id' dev1.json) \
-t mytopic/mysubtopic
# In another terminal
mosquitto_pub \
-h $(jq -r '.endpoint' hub.json) -p 8883 \
--cert dev2.crt --key dev2.key --cafile iot-hub-ca.pem \
-i $(jq -r '.device.id' dev2.json) \
-t mytopic/mysubtopic \
-m 'Hello, SECURE world!'
You can mix MQTT and MQTTs clients on the same hub.
Some additional information:
allow_insecure
flag in the devices. Same would have applied for TLS without client certificate.You can harness the real power of MQTT Pub/Sub with a few more lines of reading.
Devices information, such as events or errors, are reported on your Hub, with default topic prefix $SCW/events
.
The detailed documentation is here: IoT Hub events.
In the following example we will trigger an event by trying to connect without security on a device where security is required.
First, we connect to previously created dev1
device as events consumer (we want only error events)
# In one terminal
mosquitto_sub \
-h $(jq -r '.endpoint' hub.json) -p 1883 \
-i $(jq -r '.device.id' dev1.json) \
-t '$SCW/events/error/#'
Then we create a secured-only device and try to connect it in insecure mode:
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"hub_id": "'$(jq -r '.id' hub.json)'",
"name": "my_secured_device",
"allow_insecure": false
}' $IOT_API/devices > secure-dev.json
jq < secure-dev.json
# In another terminal
mosquitto_pub \
-h $(jq -r '.endpoint' hub.json) -p 1883 \
-i $(jq -r '.device.id' secure-dev.json) \
-t mytopic/mysubtopic \
-m 'This should fail'
dev1 will receive following message, on topic $SCW/events/device/<secure-dev-id>/error
:
{
"time":"2020-01-17T15:01:29Z",
"severity": "error",
"object-type": "device",
"object-id": "dev3-id",
"msg":"connection refused because this device requires mutual TLS authentication",
"packet":"CONNECT: dup: false qos: 0 retain: false rLength: 48"
}
Routes are integrations with the Scaleway ecosystem: they can forward MQTT messages to Scaleway services.
The documentation is there: IoT Hub Routes
This route allows you to put the payload of MQTT messages into Scaleway’s Object Storage.
This section is an addition to the “Getting started with IoT Hub” above, make sure you follow it first!
As a prerequisite, you should have s3cmd
configured for Scaleway, see guide.
BUCKET="my-bucket-$RANDOM" # Buckets are globally unique, suffix with a random number
REGION="fr-par"
PREFIX="iot/messages"
# Create the bucket
s3cmd mb --region "$REGION" "s3://$BUCKET"
# Grant write access to IoT Hub S3 Route Service to your bucket
s3cmd setacl --region "$REGION" "s3://$BUCKET" --acl-grant=write:555c69c3-87d0-4bf8-80f1-99a2f757d031:555c69c3-87d0-4bf8-80f1-99a2f757d031
# Create the IoT Hub S3 Route
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"hub_id": "'$(jq -r '.id' hub.json)'",
"name": "my-s3-route",
"topic": "hello/world",
"bucket_region": "'$REGION'",
"bucket_name": "'$BUCKET'",
"object_prefix": "'$PREFIX'",
"strategy": "per_topic"
}' $IOT_API/routes/s3 | jq
The output will contain something like:
{
"id": "e0d1e048-6f67-4c2d-8911-a75f0ef4d787",
"name": "my-s3-route",
"organization_id": "<your organization ID>",
"hub_id": "a15ddd5b-ff73-4fb3-b043-1c9176dae295",
"topic": "hello/world",
"created_at": "2018-12-13T15:17:22.052005Z",
"bucket_region": "fr-par",
"bucket_name": "my-bucket",
"object_prefix": "iot/messages",
"strategy": "per_topic"
}
Now you may publish and see the result!
sleep 5 # wait a little for the route to start
mosquitto_pub \
-h $(jq -r '.endpoint' hub.json) \
-i $(jq -r '.device.id' dev2.json) \
-t hello/world \
-m 'Hello, world!'
An object iot/messages/hello/world should now be stored in your bucket with “hello world” as contents.
s3cmd get --region "$REGION" "s3://$BUCKET/$PREFIX/hello/world"
cat world
Database route allows you to store messages in your database.
When creating such a route for one of you hubs, you need to specify a topic (wildcards are allowed), a database (with valid credentials) and a query to execute. That query may contains $TOPIC and $PAYLOAD variables.
The route will subscribe on this hub to this topic, and execute the query onto the given database for each received message:
NOTE: topic database field must be a of text type, and payload a bytea
For now, we support only PostgreSQL database system. You can use a Scaleway Database instance, or any other PostgreSQL instance publicly accessible.
This section is an addition to the “Getting started with IoT Hub” above, make sure you follow it first!
As a prerequisite, you must have a working PostgreSQL database, with valid credentials (username and password).
# Database settings
DBHOST=<your db host>
DBPORT=<your db port>
DBNAME=<your db name>
DBUSER=<your db user>
DBPASS=<your db password>
# Create the target database table
psql -h $DBHOST --port $DBPORT -U $DBUSER -d $DBNAME -c '
CREATE TABLE messages (
time timestamp,
topic text,
payload bytea
)
'
# Create the IoT Hub Database Route
# The query will insert message topic and payload with current timestamp
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"name": "my first database route",
"hub_id": "'$(jq -r '.id' hub.json)'",
"topic": "hello/world",
"query": "INSERT INTO messages VALUES (NOW(), $TOPIC, $PAYLOAD)",
"database": {
"host": "'$DBHOST'",
"port": '$DBPORT',
"dbname": "'$DBNAME'",
"username": "'$DBUSER'",
"password": "'$DBPASS'"
}
}' $IOT_API/routes/database | jq
The output will contain something like
{
"id": "486944d3-848e-4886-8945-938fbb828aa6",
"name": "my first database route",
"organizationId": "<your organization id>",
"hubId": "<your hub id>",
"topic": "hello/world",
"createdAt": "2019-10-11T13:58:42.710Z",
"query": "INSERT INTO messages VALUES (NOW(), $TOPIC, $PAYLOAD)",
"database": {
"host": "127.0.0.1",
"port": 5432,
"dbname": "route_tests",
"username": "jdoe",
"password": ""
}
}
Now you can publish a message and checks it is inserted into the message
table.
sleep 5 # wait a little for the route to start
mosquitto_pub \
-h $(jq -r '.endpoint' hub.json) \
-i $(jq -r '.device.id' dev2.json) \
-t hello/world \
-m 'Hello, world!'
psql -h $DBHOST --port $DBPORT -U $DBUSER -d $DBNAME -c "SELECT * FROM messages"
More advanced examples are available on the database route tips & tricks page.
Rest route allows you call any http(s) endpoint on received MQTT message. You can choose the HTTP verb use to call your REST uri, as well as adding extra headers.
We can see what a rest route would publish on a rest API by simply listenning to the port 80 on a public IP.
You can use a Scaleway instance, or any other machine with a public IP address.
On the machine, as root, lauch the following command:
nc -p 80 -l
Define a variable with the public IP address of your server
RESTHOST=<the_public_ip_address>
Let’s create the route
# Create the IoT Hub Rest Route
curl -sS -H "X-Auth-Token: $SCW_TOKEN" -d '{
"name": "my first rest route",
"hub_id": "'$(jq -r '.id' hub.json)'",
"topic": "hello/world",
"verb": "post",
"uri": "http://'$RESTHOST'/",
"headers": {
"X-My-Header": "Tutorial"
}
}' $IOT_API/routes/rest | jq
Now you can publish a message and checks it triggers a request on the server:
sleep 5 # wait a little for the route to start
mosquitto_pub \
-h $(jq -r '.endpoint' hub.json) \
-i $(jq -r '.device.id' dev2.json) \
-t hello/world \
-m 'Hello, world!'
On your server the above nc
output should be:
POST / HTTP/1.1
Host: <the_public_ip_address>
User-Agent: Go-http-client/1.1
Content-Length: 13
X-Mqtt-Retain: false
X-Mqtt-Topic: hello/world
X-My-Header: Tutorial
Accept-Encoding: gzip
Hello, world!