Connecting IoT Cloud Twins to Grafana Cloud

How to connect IoT Cloud Twins to Grafana Cloud

Security & Identity (IAM):

You may need certain IAM permissions to carry out some actions described on this page. This means:

you are the Owner of the Scaleway Organization in which the actions will be carried out, or
you are an IAM user of the Organization, with a policy granting you the necessary permission sets

This tutorial will show how to connect Cloud Twins to a Graphite instance on Grafana Cloud. This allows you to monitor and view events on your Cloud Twin (updates or modifications of its document’s values).

Note:

The twins feature is currently in beta status.

Requirements:

You have an account and are logged into the Scaleway console
You have installed curl and jq (json parsing tool) on your local computer
You have created an IoT Hub
You have added a device
You have created a Grafana Cloud account
Your Device allows insecure connections

Note:

When designing a production system, you should Deny Insecure connections to have the highest level of security.

Configure the following variables from a terminal on your local computer:

IOT_API="https://api.scaleway.com/iot/v1/regions/fr-par"
SCW_SECRET_KEY="<your scaleway API secret key here>"
SCW_PROJECT="<your project ID here>"
HUB_ID="<the hub_id of the hub you created>"
DEVICE_ID="<the device_id of the device you added>"

Connect your hub to Grafana Cloud

Recover your username under user identifier on the Grafana credentials page.
Generate a MetricsPublisher key and store the returned token (GRAPHITE_TOKEN).

Tip:
See Create a Grafana Cloud API key for details.
Go to your Grafana instance: https://user-identifier.grafana.net/, where user-identifier is your username.
Go to the graphite configuration panel:
- Select Configuration, then Data sources in the menu (A)
- Select Graphite on the list (B)
Get the following information from the Graphite configuration panel:
- In the HTTP box, find the URL value and note the host part (GRAPHITE_HOST, graphite-us-central1.grafana.net in the example) (A)
- In the Auth box, find the Basic Auth Details and note the user (a number as your GRAPHITE_USERID) (B)

Construct the push_uri as follows:

https://GRAPHITE_USERID:GRAPHITE_TOKEN@GRAPHITE_HOST/metrics

So it should look like:

https://12345:SecReTApiKEyVaLuE@graphite-us-central1.grafana.net/metrics

Set the graphite push_uri in your hub’s configuration:

Note:

This beta feature is not available in the console, please use the API.

curl -sS -H "X-Auth-Token: $SCW_SECRET_KEY" -X PATCH $IOT_API/hubs/$HUB_ID -d '{
  "twins_graphite_config": {
    "push_uri": "https://'$GRAPHITE_USERID':'$GRAPHITE_TOKEN'@'$GRAPHITE_HOST'/metrics"
    }
  }'

The Hub is now connected to Grafana cloud.

Using the Twin with Grafana

Now, we will consider our Cloud Twin itself. We’ll look at its document format, create a config file, and make several modifications to the document store, so that we can then see how these are logged in Grafana.

In our example the Cloud Twin’s document format is as follows:

{
    "state": {
        "enable": true,
        "value": 12.5,
        "text": "ignored"
    },
    "data": {
        "value": 7.5,
        "in error": false
    }
}

Set the configuration document

In this example we want to trace:

any write on the state (even when setting the same value twice)
changes on the data part of the document We therefore add a tag to those values in order to recover them easily in Grafana.

The _config document will be:

{
    "documents": {
        "current": {
            "state": {"_tsdb_trigger": "on-update"},
            "data": {"_tsdb_trigger": "on-change"},
            "_tags": {
                "source": "graphite-tutorial"
            }
        }
    }
}

Set the config document to the required format using the API as follows:

curl -sS -H "X-Auth-Token: $SCW_SECRET_KEY" -X PUT $IOT_API/twins/$DEVICE_ID/documents/_config -d '{
"data": {
  "documents": {
    "current": {
"state": {"_tsdb_trigger": "on-update"},
"data": {"_tsdb_trigger": "on-change"},
"_tags": {
  "source": "graphite-tutorial"
}
    }
  }
}
}' | jq

Simulate some events

Create the twin document

curl -sS -H "X-Auth-Token: $SCW_SECRET_KEY" -X PUT $IOT_API/twins/$DEVICE_ID/documents/current -d '{
"data": {
  "state": {
    "enable": false,
    "text": "ignored"
  }
}
}' | jq

Change it

curl -sS -H "X-Auth-Token: $SCW_SECRET_KEY" -X PUT $IOT_API/twins/$DEVICE_ID/documents/current -d '{
"data": {
  "state": {
    "enable": true,
    "value": 12.5,
    "text": "ignored"
  },
  "data": {
    "value": 7.5,
    "in error": false
  }
}
}' | jq

Update only the data section using patch

curl -sS -H "X-Auth-Token: $SCW_SECRET_KEY" -X PATCH $IOT_API/twins/$DEVICE_ID/documents/current -d '{
"data": {
  "data": {
    "value": 12.5
  }
}
}' | jq

Set the whole document again, without changing anything

curl -sS -H "X-Auth-Token: $SCW_SECRET_KEY" -X PUT $IOT_API/twins/$DEVICE_ID/documents/current -d '{
"data": {
  "state": {
    "enable": true,
    "value": 12.5,
    "text": "ignored"
  },
  "data": {
    "value": 12.5,
    "in error": false
  }
}
}' | jq

Show the event in Grafana

We can now return to Grafana to see how all of the modifications to our twin have been logged.

Go to your Grafana instance explore panel
- select Explore (A)
- ensure the graphite source is selected (B)
- click on the pen to switch to the query editor (C)
View your data
- enter the query (A)
```
groupByTags(seriesByTag('source=graphite-tutorial'), 'last', 'json_path', 'name')
```
- adjust the time range if needed (B)
- see your data (C)
If you prefer lines instead of dots, you can use the keepLastValue graphite function.

Replace the query with:
```
groupByTags(keepLastValue(seriesByTag('source=graphite-tutorial'), inf), 'last', 'json_path', 'name')
```

Tip:

You can create a dashboard in Grafana and use similar queries in your panel.