[IoT] Making IoT things with ESP8266/ESP32 and Deta's NoSQL Database

05 Sep, 2022

Every now and then I am tasked with creating a prototype of an IoT device as soon as possible. If the use case is simple, I often go ahead with particle. The API ecosystem is great and you can do almost everything quite quickly. The feature I love the most is the Webhooks API. I use that to tie in some microservices and do remote firmware updates.

But sometimes...

The prototype needs to be battery operated and even last for months on the battery. The form factor has to be designed around the battery and then, I move away from Particle. This is because :

1. I have to pay a lot of customs in Spain when importing. The customs office here still uses pigeons to send papers between departments. On average it takes about a month to get the devices in hand, minus pecking breaks.
2. I am usually not making PCBs at this stage. So, I want something that is available in many form-factors. This provides me with options for my product size. (ESP chips are available in several form factors)
3. My knowledge of what goes inside particle is quite limited at the moment. Sometimes, I cannot find easy ways to "get things done". (ESP ecosystem has great community documentation)

So, yeah! ESP8266/32 wins in my use-case this time.

With this off my chest, Let us talk about a way to connect ESP8266 to my favourite cloud service (also Heroku killed its free tier, which I used for prototyping) — Deta.

Most IoT devices have an I for Internet component. It can be MQTT or something like a mesh of server-less functions (This makes Julian laugh all the time). For quick and dirty experience tests, I usually go ahead with a serverless approach.

In this particular post I will describe (to my future self) how to read stored data in Deta's NoSQL database using an ESP8266. I do not prefer, writing the data from my ESP device to the database. This is because :

• Most of the time, I need to perform some operation on the data before storing it, like adding timestamps etc.
• Writing to the database from insecure IoT sensors casn affect the data integrity of the application. In my case, the database is the single source of truth.

That being said, you can always write to the database, at your risk.

Deta exposes a HTTP API to access the database. You can use it to do all things CRUD. The important thing to note is that the connection has to be over HTTPS. There are several ways to do so as illustrated in this wonderful article. I chose the least secure one — client.setInsecure(). Obviously, do not use this in production. But hey! My job involves making prototypes AFAP.

So, we connect to the database over an HTTPS connection, read the data, and go to sleep for a predetermined time. The code below should do that. You will need the following information from your Deta dashboard to begin: Project ID, API Key, and Key of the entry you want to read (I usually read a predefined key).

The documentation of deta is quite good to expand this example to do cool things like writing to the database or updating values.

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ESP8266WebServer.h>
#include <ESP8266HTTPClient.h>
#include <ArduinoJson.h>

const char *ssid = "your wifi";
const char *password = "your password";

const char *host = "database.deta.sh";
const int httpsPort = 443; // HTTPS= 443 and HTTP = 80

DynamicJsonDocument doc(1024);

void setup()
{
  delay(1000);
  Serial.begin(115200);

  WiFi.mode(WIFI_OFF); // Prevents reconnection issue (taking too long to connect)
  delay(1000);
  WiFi.mode(WIFI_STA); // Only Station No AP, This line hides the viewing of ESP as a wifi hotspot

  Serial.println("");

  Serial.print("Connecting");
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  WiFiClientSecure httpsClient; // Declare object of class WiFiClient

  Serial.println(host);

  httpsClient.setInsecure();
  httpsClient.setTimeout(15000); // 15 Seconds
  delay(1000);

  Serial.print("HTTP Connecting");
  int r = 0; // retry counter
  while ((!httpsClient.connect(host, httpsPort)) && (r < 30))
  {
    delay(100);
    Serial.print(".");
    r++;
  }
  if (r == 30)
  {
    Serial.println("Connection failed");
  }
  else
  {
    Serial.println("Connected to web");
  }

  String Link;
  // GET Data
  Link = "/v1/[Project ID]/[Database Name]/items/[Key]";

  Serial.print("requesting URL: ");
  Serial.println(host + Link);

  httpsClient.print(String("GET ") + Link + " HTTP/1.1\r\n" +
                    "Host: " + host + "\r\n" +
                    "Content-Type: " + "application/json" + "\r\n" +
                    "X-API-Key: " + "[API-Key]" + "\r\n" +
                    "Connection: close\r\n\r\n");

  Serial.print("request sent");
  Serial.println(millis());
  while (httpsClient.connected())
  {
    String line = httpsClient.readStringUntil('\n');
    if (line == "\r")
    {
      Serial.println("headers received");
      break;
    }
  }
  Serial.println("reply was:");
  Serial.println("==========");
  String line;
  while (httpsClient.available())
  {
    line = httpsClient.readStringUntil('\n'); // Read Line by Line
    Serial.println(line);                     // Print response
  }
  Serial.println("==========");
  Serial.println("closing connection");
  unsigned int startIndex = line.indexOf('{');
  unsigned int endIndex = line.lastIndexOf('}');
  String input = line.substring(startIndex, endIndex);
  Serial.println(input);
  deserializeJson(doc, input);
  JsonObject obj = doc.as<JsonObject>();

  unsigned int data = obj["some key"];
  Serial.println(data);
  delay(1000);

  Serial.println("Now Sleeping zzZZzzZZZZ");
  Serial.print("END:");
  ESP.deepSleep(27e6); // 27e6 is 27 seconds
}

//=======================================================================
//                    Main Program Loop
//=======================================================================
void loop()
{
}
//=======================================================================

The data output is JSON and it is parsed using the ArduinoJSON library. The deep sleep wakeup works only when you connect the pin D0 to RST.

Hope this helps you prototype faster. My name is Rohit and if you have any issues running this, send me a message on Twitter.

In a future post, I will talk about sending API requests to Deta Micros (like AWS Lamba or Netlify Functions, but kinder).

Cheers, Rohit

Notes :

1. One can actually expand this to work with Particle too. You can use Particle's Webhook API.
2. Even though I use ESP8266 here, this can probably work with ESP32 devices too.
3. The code is written for Arduino environments inside PlatformIO.

I usually write one a month. Subscribe to my blog via email or RSS feed.

#iot #webdev