Monitoring Sensor’s Data in the Air using Augmented Reality

ESP32 Projects, IoT Projects

We at techiesms studios, running a Series called AR with IoT and this is the fourth episode of that series. So till now we were easily able to control our Home Appliances over internet using Augmented Reality and Internet Of Things both using touch screen buttons and virtual buttons.

Now in this episode, we will try to read data from server and display that data in the Air using Augmented Reality.

Components Required

For Hardware part of the project you’ll need 

  • ESP32 / NodeMCU board
  • DHT11 Sensor

For Software part you’ll need 

  • Blynk App
  • Unity Hub

Hardware Connection Diagram

IoT part of the project 

For the IoT part, I have used Blynk IoT Platform. In which, I just made a simple blynk project of displaying Temperature and Humidity data using Gauge widget. Here the temperature data is stored in virtual pin V0 and humidity data is stored in virtual pin V1

After that, we have made a code in which, the temperature data from DHT11 sensors gets pushed to virtual pin V0 and Humidity data is pushed to Virtual pin V1. You can copy and paste the code from below, just change the SSID name, Password and Blynk Auth token. 

/*************************************************************
  Download latest Blynk library here:
    https://github.com/blynkkk/blynk-library/releases/latest

  Blynk is a platform with iOS and Android apps to control
  Arduino, Raspberry Pi and the likes over the Internet.
  You can easily build graphic interfaces for all your
  projects by simply dragging and dropping widgets.

    Downloads, docs, tutorials: http://www.blynk.cc
    Sketch generator:           http://examples.blynk.cc
    Blynk community:            http://community.blynk.cc
    Follow us:                  http://www.fb.com/blynkapp
                                

  Blynk library is licensed under MIT license
  This example code is in public domain.

 *************************************************************

  This example shows how value can be pushed from Arduino to
  the Blynk App.

  WARNING :
  For this example you'll need Adafruit DHT sensor libraries:
    https://github.com/adafruit/Adafruit_Sensor
    https://github.com/adafruit/DHT-sensor-library

  App project setup:
    Value Display widget attached to V5
    Value Display widget attached to V6
 *************************************************************/

/* Comment this out to disable prints and save space */
#define BLYNK_PRINT Serial

/* Fill-in your Template ID (only if using Blynk.Cloud) */
//#define BLYNK_TEMPLATE_ID   "YourTemplateID"

#include <WiFi.h>
#include <WiFiClient.h>
#include <BlynkSimpleEsp32.h>
#include <DHT.h>

// You should get Auth Token in the Blynk App.
// Go to the Project Settings (nut icon).
char auth[] = "YourAuthToken";

// Your WiFi credentials.
// Set password to "" for open networks.
char ssid[] = "YourNetworkName";
char pass[] = "YourPassword";

#define DHTPIN 2          // What digital pin we're connected to

// Uncomment whatever type you're using!
#define DHTTYPE DHT11     // DHT 11
//#define DHTTYPE DHT22   // DHT 22, AM2302, AM2321
//#define DHTTYPE DHT21   // DHT 21, AM2301

DHT dht(DHTPIN, DHTTYPE);
BlynkTimer timer;

// This function sends Arduino's up time every second to Virtual Pin (5).
// In the app, Widget's reading frequency should be set to PUSH. This means
// that you define how often to send data to Blynk App.
void sendSensor()
{
  float h = dht.readHumidity();
  float t = dht.readTemperature(); // or dht.readTemperature(true) for Fahrenheit

  if (isnan(h) || isnan(t)) {
    Serial.println("Failed to read from DHT sensor!");
    return;
  }
  // You can send any value at any time.
  // Please don't send more that 10 values per second.
  Blynk.virtualWrite(V1, h);
  Blynk.virtualWrite(V0, t);
}

void setup()
{
  // Debug console
  Serial.begin(9600);

  Blynk.begin(auth, ssid, pass);

  dht.begin();

  // Setup a function to be called every second
  timer.setInterval(1000L, sendSensor);
}

void loop()
{
  Blynk.run();
  timer.run();
}

Till this step, we will be easily able to monitor sensor’s data on the Blynk App. But we want to monitor that using API so that later on we can integrate it with AR. And the good news is that, Blynk do provide us APIs. 

So to get the data to virtual pin of our blynk device, the format of API is something like this.

http://blynk-cloud.com/auth_token/get/pin

For example, If I want to read data from virtual pin ‘V1’ of my blynk project, the API will be like this,

http://blynk-cloud.com/MY_AUTH_TOKEN/get/V1

So with this, we have successfully covered the IoT part of our project. Now let’s jump on to the AR part. 

AR part of project 

For AR, we will be using Unity Hub software on our computer. Just goto this link and download unity hub

While Setting up unity, you’ll need four things, one is the target image, second is Button Image, third is Template Image and fourth is C# Script. All the things are provided below, so you can easily download and use them

Target Image
Button Image
Template Image
using System.Collections;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.Networking;
using Vuforia;

public class click : MonoBehaviour
{
    InputField field;
    InputField Hum;
    public VirtualButtonBehaviour Vb_on;
 
    void Start()
    {
        field = GameObject.Find("TextInputField").GetComponent<InputField>();
        
        Hum = GameObject.Find("InputField1").GetComponent<InputField>();

        Vb_on.RegisterOnButtonPressed(OnButtonPressed_on);
        // GameObject.Find("GetButton").GetComponent<Button>().onClick.AddListener(GetData);
    }

    public void OnButtonPressed_on(VirtualButtonBehaviour Vb_on)
    {
        GetData_tem();
        GetData_hum();
        Debug.Log("Click");
    }
 
    void GetData_tem() => StartCoroutine(GetData_Coroutine1());
    void GetData_hum() => StartCoroutine(GetData_Coroutine());
 
    IEnumerator GetData_Coroutine1()
    {
        Debug.Log("Getting Data");
        field.text = "Loading...";
        string uri = "http://blynk-cloud.com/vKqIp55UdG2GoZcvY3un4sPyQpoxgnG3/get/v0";
        using(UnityWebRequest request = UnityWebRequest.Get(uri))
        {
            yield return request.SendWebRequest();
            if (request.isNetworkError || request.isHttpError)
                field.text = request.error;
            else
            {

                field.text = request.downloadHandler.text;
                field.text = field.text.Substring(2,2);
            }
        }
    }
    IEnumerator GetData_Coroutine()
    {
        Debug.Log("Getting Data");
        Hum.text = "Loading...";
        string uri = "http://blynk-cloud.com/vKqIp55UdG2GoZcvY3un4sPyQpoxgnG3/get/v1";
        using(UnityWebRequest request = UnityWebRequest.Get(uri))
        {
            yield return request.SendWebRequest();
            if (request.isNetworkError || request.isHttpError)
                Hum.text = request.error;
            else
            {

                Hum.text = request.downloadHandler.text;
                Hum.text = Hum.text.Substring(2,2);
            }
        }
    }
}

You need to save this code with the name, Click.cs

Full Tutorial 

We have prepared a full detailed tutorial video for AR+IOT which is uploaded on YouTube. So in case you want to understand it in more clear manner, I’ll suggest you to go through this video and do let me know your feedback in comment section.

Sachin Soni

techiesms