Google Assistant(4/4):  Voice-Activated Sensor Reading and Light Control using NodeMCU and Raspberry Pi with Google Assistant and Dialogflow

Overview

This tutorial is the last in the 4-part series on how to set up Google Assistant with the Raspberry Pi. In this tutorial you will learn how to control a relay switch connected to a light bulb and read temperature with DHT22 sensor using web services and tools like Dialogflow, Webhook, Flask, NGROK, and Google Assistant. The hardware we're using are NodeMCU, Raspberry Pi and ReSpeaker 4-Mics Pi Hat. Of course, we're going to control the light turning on and off using voice commands, as well as ask Google about current temperature on our sensor :D

 

Hardware

 

Software

  • Arduino IDE
  • RPI terminal
  • Internet Browser (Example Chrome)

 

Application Discussion

We basically know that to turn ON and OFF a light bulb, we need a switch connected to its electrical wire. To control it remotely and online, we need a microcontroller to do that.

DHT22 is a basic, low-cost digital temperature and humidity sensor. It uses a capacitive humidity sensor and a thermistor to measure the surrounding air, and spits out a digital signal on the data pin (no analog input pins needed). It's fairly simple to use, but requires careful timing to grab data. The only real downside of this sensor is you can only get new data from it once every 2 seconds.

In this tutorial we will use the NodeMCU Wi-fi development board as the microcontroller and a relay module to control the light.

NodeMCU is a low-cost open-source IoT platform. It has firmware which runs on the ESP8266 Wi-Fi SoC from Espressif Systems and hardware which was based on the ESP-12 module. It has support for Arduino IDE programming.

We need to setup NodeMCU as a web client and connect it to Google Assistant.

Google Assistant is an artificial intelligence-powered virtual assistant developed by Google. With the integration of the Raspberry Pi and ReSpeaker device, we can make a portable Google Assitant device that enables voice-activated searches, playing music, and so on.

In order to connect NodeMCU to Google Assistant we will use Dialogflow.

 

Dialogflow is a natural language understanding platform that makes it easy to design and integrate a conversational user interface into your mobile app, web application, device, bot, interactive voice response system, and so on. Using Dialogflow, you can provide new and engaging ways for users to interact with devices.

Dialogflow can analyze multiple types of input including text or audio (like from a phone or voice recording). It can also respond in a couple of ways, either through text or with synthetic speech.

To connect NodeMCU web server to DialogFlow, we will use Dialogflow Webhooks service.

 

Webhooks is an HTTP request that is sent automatically whenever certain criteria is fulfilled. A webhook can be created in any server side programming language like Python, PHP or Node.js. You can read more about webhooks here.

In Dialogflow, a webhook can be used to fetch data from your server whenever a certain intent having webhook enabled is invoked. The information from the intent is passed to the webhook service to receive result.

We need a web server to run a web application on our Raspberry Pi. We will use Flask to implement this local web application.

 

Flask is a web framework. It’s a Python module that lets you develop web applications easily. We will use this to implement our local web server in our Raspberry Pi.

To access our Flask local web server through the internet, we will use NGROK for local tunneling to the internet. No need for complicated setups or port forwarding.

ngrok makes our locally-hosted web server (e.g. Raspberry Pi) appear to be hosted on a subdomain of ngrok.com, meaning that no public IP or domain name on the local machine is needed.

To connect NodeMCU to Flask local web application, we will use HTTP Requests. HTTP Requests is how a packet of data is sent by the Client to Server.

 

 

 

Setup the Hardware

NodeMCU Setup

Follow the schematic diagram below.

CAUTION: Special care must be taken when working with high AC Voltage!!

 

Google Assistant on Raspberry Pi Setup

Mount the ReSpeaker 4-Mics Pi HAT to Raspberry pi like this.

 

Setup the Software

Google Assistant Setup

  • To set up the Google Assistant with Raspberry Pi, follow the first tutorial. You can check it here.

Raspberry Pi with ReSpeaker 4-Mic Pi Hat Integration

 

NodeMCU with Arduino IDE Setup

  • First, we will program the NodeMCU. We need to download and install the newest version of Arduino IDE, you can download it here.

https://www.arduino.cc/en/software

  • In the Arduino IDE, go to Preferences. You can press "CTRL+Comma", or click "File" then "Preferences" as shown in the screenshot below.

 

  • On the Preferences window, copy the text below and paste it on "Additional Boards Manager URLs" field. This is for adding the ESP8266 board driver. Then click "OK".

http://arduino.esp8266.com/stable/package_esp8266com_index.json

 

  • Proceed to download the ESP8266 library on the Boards Manager. Click "Tools", then "Boards:(Your Current Board)", and "Boards Manager...". 

 

  • In the "Boards Manager" first select "All" on the Type drop-down box, then search for esp8266. Download the esp8266 by ESP8266 Community, by selecting the latest version and clicking Install. After installation, you may close it.

 

void setup() {
  pinMode(LED_BUILTIN, OUTPUT);     // Initialize the LED_BUILTIN pin as an output
}

// the loop function runs over and over again forever
void loop() {
  digitalWrite(LED_BUILTIN, LOW);   // Turn the LED on (Note that LOW is the voltage level
  // but actually the LED is on; this is because
  // it is active low on the ESP-01)
  delay(1000);                      // Wait for a second
  digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
  delay(2000);                      // Wait for two seconds (to demonstrate the active low LED)
}  
  • Paste it to your Arduino code editor (or open the downloaded file), then click Upload by clicking the -> icon as shown below.

  • If you see your NodeMCU's built-in LED blinking, we're good at setting up our NodeMCU. Next is to test our Relay and DHT22 sensor setup.

 

Relay and DHT22 Setup

  • First, we need to create a new sketch. Press "CTRL+N", or click "File" on the top left side and then click "New".



  • After creating a new sketch download the DHT library. To do that go to "Sketch" then click "Include Library" then click "Manage Libraries".



  • Then search for "DHT sensor library", look for "DHT sensor library" by Adafruit then hover to it and click the "Install" button.

// Relay State Changing and DHT22 Test
// Written by jeminico (Createlabz)

/************************ Relay Configuration *******************************/
// Relay pin
int relayPin = 15;
// Relay State
int state = LOW;
// Timer delay for changing state
long relayTimer;
const long relayTimerDelay = 5000;

/************************ DHT Sensor Configuration *******************************/
// Example testing sketch for various DHT humidity/temperature sensors
// Written by ladyada, public domain

// REQUIRES the following Arduino libraries:
// - DHT Sensor Library: https://github.com/adafruit/DHT-sensor-library
// - Adafruit Unified Sensor Lib: https://github.com/adafruit/Adafruit_Sensor
#include "DHT.h"

 // Digital pin connected to the DHT sensor
#define DHTPIN 4    
// Feather HUZZAH ESP8266 note: use pins 3, 4, 5, 12, 13 or 14 --
// Pin 15 can work but DHT must be disconnected during program upload.

// Uncomment whatever type you're using!
//#define DHTTYPE DHT11   // DHT 11
#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
//#define DHTTYPE DHT21   // DHT 21 (AM2301)

// Connect pin 1 (on the left) of the sensor to +5V
// NOTE: If using a board with 3.3V logic like an Arduino Due connect pin 1
// to 3.3V instead of 5V!
// Connect pin 2 of the sensor to whatever your DHTPIN is
// Connect pin 4 (on the right) of the sensor to GROUND
// Connect a 10K resistor from pin 2 (data) to pin 1 (power) of the sensor

// Initialize DHT sensor.
// Note that older versions of this library took an optional third parameter to
// tweak the timings for faster processors.  This parameter is no longer needed
// as the current DHT reading algorithm adjusts itself to work on faster procs.
DHT dht(DHTPIN, DHTTYPE);
float humid,tempCelc,tempFaren;
// Timer delay for reading temperature
long tempTimer;
const long tempTimerDelay = 2000;


void setup() {
  // put your setup code here, to run once:
  // Initialize Serial
  Serial.begin(9600);

  // Initialize relay
  Serial.println("Initializing Relay..");
  pinMode(relayPin, OUTPUT);
  
  // Initialize DHT Sensor
  Serial.println("Initializing DHT Sensor..");
  dht.begin();

   // Print done initialize
   Serial.print("Done initializing.. ");
}

void loop() {
  // put your main code here, to run repeatedly:
  readTemp();
  changeStateRelay();
}

// Read Temperature
void readTemp(){
  // Timer to read temperature
  if(millis() <= tempTimer + tempTimerDelay) return;
  tempTimer = millis();
  // Reading temperature or humidity takes about 250 milliseconds!
  // Sensor readings may also be up to 2 seconds 'old' (its a very slow sensor)
  humid = dht.readHumidity();
  // Read temperature as Celsius (the default)
  tempCelc = dht.readTemperature();
  // Read temperature as Fahrenheit (isFahrenheit = true)
  tempFaren = dht.readTemperature(true);

  // Check if any reads failed and exit early (to try again).
  if (isnan(humid) || isnan(tempCelc) || isnan(tempFaren)) {
    Serial.println(F("Failed to read from DHT sensor!"));
    return;
  }

  // Compute heat index in Fahrenheit (the default)
  float hif = dht.computeHeatIndex(tempFaren, humid);
  // Compute heat index in Celsius (isFahreheit = false)
  float hic = dht.computeHeatIndex(tempCelc, humid, false);

  // Display DHT Sensor reading values
  Serial.print(F("Humidity: "));
  Serial.print(humid);
  Serial.print(F("%  Temperature: "));
  Serial.print(tempCelc);
  Serial.print(F("°C "));
  Serial.print(tempFaren);
  Serial.print(F("°F  Heat index: "));
  Serial.print(hic);
  Serial.print(F("°C "));
  Serial.print(hif);
  Serial.println(F("°F"));
}

// Change state of the Relay
void changeStateRelay(){
  // Timer to change state
  if(millis() <= relayTimer + relayTimerDelay) return;
  relayTimer = millis();
  
  // Set state of relay
  if(state == LOW) state = HIGH;
  else state = LOW;
  digitalWrite(relayPin, state);
  Serial.print("Relay Change State to ");
  Serial.println(state);
}

  • Paste it to your Arduino code editor (or open the downloaded file), then Upload to your NodeMCU.
  • After successful upload, open Serial Monitor by pressing "CTRL+SHIFT+M", or click "Tools" then "Serial Monitor" . You can see the serial logs printed on the serial monitor as shown below. 
  • On the serial monitor, you can see the temperature and humidity values every 2 seconds, and the shange in state of relay every 5 seconds. Also, you can observe the light bulb is turning on and off every 5 seconds. Next is to proceed to setup NodeMCU web server.

 

NodeMCU Web Server with DHT22 and Relay Setup

// Load Wi-Fi library
#include 
/************************ Wifi Server Configuration *******************************/
// Replace with your network credentials
const char* ssid     = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";

// Set web server port number to 80
WiFiServer server(80);
// Variable to store the HTTP request
String header;
// Current time
unsigned long currentTime = millis();
// Previous time
unsigned long previousTime = 0; 
// Define timeout time in milliseconds (example: 2000ms = 2s)
const long timeoutTime = 2000;

// Timer to display local IP on serial monitor.
long getIPTimer;
long getIPTimerDelay = 5000;

/************************ Relay Configuration *******************************/
// Relay pin
int relayPin = 15;

/************************ DHT Sensor Configuration *******************************/
// Example testing sketch for various DHT humidity/temperature sensors
// Written by ladyada, public domain

// REQUIRES the following Arduino libraries:
// - DHT Sensor Library: https://github.com/adafruit/DHT-sensor-library
// - Adafruit Unified Sensor Lib: https://github.com/adafruit/Adafruit_Sensor
#include "DHT.h"

 // Digital pin connected to the DHT sensor
#define DHTPIN 4    
// Feather HUZZAH ESP8266 note: use pins 3, 4, 5, 12, 13 or 14 --
// Pin 15 can work but DHT must be disconnected during program upload.

// Uncomment whatever type you're using!
//#define DHTTYPE DHT11   // DHT 11
#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
//#define DHTTYPE DHT21   // DHT 21 (AM2301)

// Connect pin 1 (on the left) of the sensor to +5V
// NOTE: If using a board with 3.3V logic like an Arduino Due connect pin 1
// to 3.3V instead of 5V!
// Connect pin 2 of the sensor to whatever your DHTPIN is
// Connect pin 4 (on the right) of the sensor to GROUND
// Connect a 10K resistor from pin 2 (data) to pin 1 (power) of the sensor

// Initialize DHT sensor.
// Note that older versions of this library took an optional third parameter to
// tweak the timings for faster processors.  This parameter is no longer needed
// as the current DHT reading algorithm adjusts itself to work on faster procs.
DHT dht(DHTPIN, DHTTYPE);
float humid,tempCelc,tempFaren;

void setup() {
  // Initialize Serial
  Serial.begin(9600);
  
  // Initialize relay
  Serial.println("Initializing Relay..");
  pinMode(relayPin, OUTPUT);

  // Initialize DHT Sensor
  Serial.println("Initializing DHT Sensor..");
  dht.begin();
  
  // Initialize WiFi Server
  Serial.print("Initializing WiFi Server..");
  // Connect to Wi-Fi network with SSID and password
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  // Print local IP address and start web server
  Serial.println("");
  Serial.println("WiFi connected.");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  server.begin();

  // Print done initialize
   Serial.print("Done initializing.. ");
}

void loop(){
  WiFiClient client = server.available();   // Listen for incoming clients

  if (client) {                             // If a new client connects,
    Serial.println("New Client.");          // print a message out in the serial port
    String currentLine = "";                // make a String to hold incoming data from the client
    currentTime = millis();
    previousTime = currentTime;
    while (client.connected() && currentTime - previousTime <= timeoutTime) { // loop while the client's connected
      currentTime = millis();         
      if (client.available()) {             // if there's bytes to read from the client,
        char c = client.read();             // read a byte, then
        Serial.write(c);                    // print it out the serial monitor
        header += c;
        if (c == '\n') {                    // if the byte is a newline character
          // if the current line is blank, you got two newline characters in a row.
          // that's the end of the client HTTP request, so send a response:
          if (currentLine.length() == 0) {
            
            String deviceName;
            String deviceStatus;
            // turns the GPIOs on and off
            if (header.indexOf("GET /light/on") >= 0) {
              Serial.println("Light on");
              deviceName = "Light";
              deviceStatus = "on";
              digitalWrite(relayPin, HIGH);
            } else if (header.indexOf("GET /light/off") >= 0) {
              Serial.println("Light off");
              deviceName = "Light";
              deviceStatus = "off";
              digitalWrite(relayPin, LOW);
              // Set here
            } else if (header.indexOf("GET /sensor/dht22/temperature/read/celcius") >= 0) {
              readTemp();
              Serial.println("DHT22 Sensor read");
              Serial.print("Temp Read");
              Serial.println(tempCelc);
              deviceName = "DHT22";
              deviceStatus = String(tempCelc) + " in celcius";
            } 
            // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
            // and a content-type so the client knows what's coming, then a blank line:
            client.println("HTTP/1.1 200 OK");
            client.println("Content-Type: application/json");
            client.println("Access-Control-Allow-Origin: *");
            client.println("");
            // Send JSON response
            client.print("{\""+deviceName+"\": \""
            +deviceStatus+"\"}");
            // The HTTP response ends with another blank line
            client.println();
            // Break out of the while loop
            break;
          } else { // if you got a newline, then clear currentLine
            currentLine = "";
          }
        } else if (c != '\r') {  // if you got anything else but a carriage return character,
          currentLine += c;      // add it to the end of the currentLine
        }
      }
    }
    // Clear the header variable
    header = "";
    // Close the connection
    client.stop();
    Serial.println("Client disconnected.");
    Serial.println("");
  }
  
  getIPAddress();
}

// Read Temperature
void readTemp(){
  // Reading temperature or humidity takes about 250 milliseconds!
  // Sensor readings may also be up to 2 seconds 'old' (its a very slow sensor)
  humid = dht.readHumidity();
  // Read temperature as Celsius (the default)
  tempCelc = dht.readTemperature();
  // Read temperature as Fahrenheit (isFahrenheit = true)
  tempFaren = dht.readTemperature(true);

  // Check if any reads failed and exit early (to try again).
  if (isnan(humid) || isnan(tempCelc) || isnan(tempFaren)) {
    Serial.println(F("Failed to read from DHT sensor!"));
    return;
  }

  // Compute heat index in Fahrenheit (the default)
  float hif = dht.computeHeatIndex(tempFaren, humid);
  // Compute heat index in Celsius (isFahreheit = false)
  float hic = dht.computeHeatIndex(tempCelc, humid, false);

  // Display DHT Sensor reading values
  Serial.print(F("Humidity: "));
  Serial.print(humid);
  Serial.print(F("%  Temperature: "));
  Serial.print(tempCelc);
  Serial.print(F("°C "));
  Serial.print(tempFaren);
  Serial.print(F("°F  Heat index: "));
  Serial.print(hic);
  Serial.print(F("°C "));
  Serial.print(hif);
  Serial.println(F("°F"));
}

void getIPAddress(){
  // Delay display ip address
  if(millis() <= getIPTimer + getIPTimerDelay) return;
  getIPTimer = millis();
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

  • Paste it to your Arduino code editor (or open the downloaded file).
  • Find the part for setting up the WiFi credentials as shown below, and replace the value of  "YOUR_WIFI_SSID" with your actual WiFI SSID, and "YOUR_WIFI_PASSWORD" with your actual Wifi password.
// Replace with your network credentials
const char* ssid     = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";
  • After setting your WiFi Credentials, you can Upload the Arduino code (save it after upload if new sketch) to your NodeMCU.
  • After successfully uploading, go to Serial Monitor by pressing "CTRL+SHIFT+M", or click "Tools" then "Serial Monitor". You can see the serial logs printed on the serial monitor as shown below.
  • On the serial monitor, you can see the IP Address of your NodeMCU web server, take note of the IP Address, we will need it later.
  • Now go to your internet browser, enter this URL "http://{your_ip_address}/light/on" to turn the light on, you need to replace {your_ip_address} to your NodeMCU IP address. In my case my IP Address is 10.0.254.4, so the URL will be "http://10.0.254.4/light/on". 
  • If no problems, you will receive the JSON-formatted reply on the web page as shown below.

  • Now for turning the light off search this URL "http://{your_ip_address}/light/off"you need to replace {your_ip_address} to your NodeMCU IP address. And see the JSON-formatted reply.

  • Now to read the temperature of the DHT22 sensor search this URL "http://{your_ip_address} sensor/dht22/temperature/read/celcius"you need to replace {your_ip_address} to your NodeMCU IP address. And see the JSON-formatted reply.

  • After successfully testing it, we can now go to creating a Flask web application.

 

Python Flask Web Application Setup

  • Now go to your Raspberry Pi terminal and input these commands.
# Back to user directory
$ cd ~
# Update and upgrade Raspberry Pi OS and packages $ sudo apt-get update $ sudo apt-get upgrade
# Install needed packages $ source env/bin/activate $ pip install requests $ pip install -U Flask $ deactivate
# Clone repository
$ sudo apt-get install git $ git clone https://github.com/createlabz/createlabz-public-raspberry-pi
  • For the codes, you can check from the CreateLabz Github Raspberry Pi Repository on "light-dht22-nodemcu-dialogflow-webhook" here.

https://github.com/createlabz/createlabz-public-raspberry-pi/tree/master/light-dht22-nodemcu-dialogflow-webhook

  • To run the codes input the command below.
$ /home/${USER}/env/bin/python -u /home/${USER}/createlabz-public-raspberry-pi/light-dht22-nodemcu-dialogflow-webhook/src/webhookDialogflow.py "NODEMCU_IP_ADDRESS"
  • And replace the "NODEMCU_IP_ADDRESS" with your NodeMCU's local IP address. For example, my local IP Address is "http://10.0.254.4/" so my command will be:
    $ /home/${USER}/env/bin/python -u /home/${USER}/createlabz-public-raspberry-pi/light-dht22-nodemcu-dialogflow-webhook/src/webhookDialogflow.py "http://10.0.254.4/"
  • Please Note:
  1. Don't forget to add / at the end of the URL
  2. Every time the router will reset or NodeMCU disconnects to network, the NodeMCU's local IP address will change because its currently programmed for dynamic IP address
  • Now your Flask web application is running and this will be displayed on your terminal. (Don't close this terminal, if you close this the Flask web application will be terminated)

  • Now test your Flask web application, if you're not remotely connected to your Raspberry PI just open an internet browser (e.g. Chromium) and search for "localhost:8080". And a "Hello World!" text will be displayed on the web page.

  • If you want to access your RPi Flask web application from another device (as long as within the same network with your RPi), need to get Raspberry Pi's local IP address first. To do that open a new terminal (Don't close the current terminal), and input the command.
$ ifconfig
  • You will be shown network connection details of your RPi like IP address. Look for the inet value, which is the IP address of your Raspberry Pi. For this tutorial, I'm using a WiFi connection so I'm looking for the details of the wlan0 connection and my inet (RPI Local Address) value will be shown like this. If you're using an ethernet connection look for the eth0 details.

  • Now you got your Raspberry Pi IP address, open a web browser on your device, and enter this URL "{your_ip_address}:8080". For example, my URL will be "10.0.254.201:8080". And a "Hello World!" text will be displayed on the web page as shown below.

 

  • After successful test, we will tunnel our local Flask web application to the internet using ngrok. This will allow us to access our Raspberry Pi from anywhere or any web service outside our local network.

 

ngrok setup

  • First, open a web browser then search for this URL.

https://dashboard.ngrok.com/login

  • Then sign in or create an account, after signing in you will be shown this web page. Take note of the authtoken as indicated below, we will need it later.

  • Next is to install ngrok in your Raspberry Pi. Open a new terminal, then input this command. In the command below replace authtoken_value with your authtoken.
$ sudo apt install snapd
$ sudo snap install ngrok
$ ngrok authtoken authtoken_value
$ ngrok http http://0.0.0.0:8080
  • If you encounter ths error: "listen tcp 127.0.0.1:4049: listen: operation not permitted", reboot your Raspberry Pi , and wait a couple of minutes to input these command again. (This is an issue with the current ngrok). If not proceed to next step
$ ngrok http http://0.0.0.0:8080
  • Open new terminal to run the Flask web application.
   $ /home/${USER}/env/bin/python -u /home/${USER}/createlabz-public-raspberry-pi/light-dht22-nodemcu-dialogflow-webhook/src/webhookDialogflow.py "NODEMCU_IP_ADDRESS"
  • After a successful run, copy the ngrok HTTPs URL, open a web browser and search it. A "Hello World!" text will be displayed if successfully tunneled. Take note of the ngrok HTTPs URL. we will need it later. Now we will setup DialogFlow.
Dialogflow Setup
  • Open a new tab on your web browser, then enter this:

https://dialogflow.cloud.google.com/

  • If not already signed in, sign in with your Google Assistant. After you sign in, you will be directed to this web page.

  • Next click on the button "+ Create Agent" on the top left side. On the "Google Project" drop-down field select your Google Assistant with Raspberry Pi project. For example, see screenshot below. Then press "Create" button.

  • Go to "Entities" tab as shown below. Click the button "CREATE ENTITY".

  • Then we will create a "device_nameentity, to create the entity fill up the fields as shown below. Then click the "Save" button on the top right side.

  • Click the "CREATE ENTITY" button again. This time we will create a "device_status" entity,  to create the entity fill up the fields as shown below. Then click "Save".

  • After saving, click the "Intents" tab as shown below. 

  • Then on the "Intent name" field input the value device_control as shown below. Then click the "ADD TRAINING PHRASES".

  • For the light on, Add the training phrase "light on" then press "Enter".

It will automatically add the entity parameter we created earlier as shown below.

  • Then Add the training phrase "light off" for turning the light off as shown below, then press "Enter".

It will automatically add the entity parameter we created earlier as shown below.

  • Then scroll down and look for the "Responses" section and "Text Response" field. Add a value on the field or you may copy our example shown below. This is the default response on "light on" and "light off" phrases.

  • Then click "SAVE" button on the top right.
  • Test by typing "light on" on the "try it now" text field on the upper right side column and press "Enter" as shown below.

Then this will be displayed.

  • Then type "light off" again on the text field for turning off the light.
  • Now we will proceed to integrate our Flask web application with ngrok HTTPS URL to Dialogflow webhook service. To do that go to "Fulfillment", then click the "disabled" switch to enable Webhook as shown below. 

  • After enabling, provide your ngrok HTTPS URL we noted earlier and add "/webhook" endpoint after the URL (URL: https://{ngrok HTTPS URL}/webhook ) Example shown below. (Note: every time the terminal is closed, ngrok will be terminated, and if run again the URL will change so take note of the ngrok HTTPS URL again and change this value due to the free version.)

  • Scroll down, and click the "Save" button.

  • Next, go back to the "Intents" tab. Click "device_control" as shown below.

  • Next, scroll down and click "Enable Fulfillment" as shown below.

  • Click the switch "Enable webhook call for this intent" to enable as shown below. Then click the button "Save".

  • Test it again. For turning the light on, type "light on" on the "try it now" field.  As you notice the light bulb will turn on and the default response will change from "Light has been turned as you ask.." to "light has been turned on" as shown below.

  • For turning the light off, type "light off" on the "try it now" field. Then the light bulb will turn off and the default response will change as shown below. 

  • Now the NodeMCU web server is connected to DialogFlow. You can add other training phrases like "turning the light on" and more, on the "Training phrases" section. 
  • Now we will create an intent for reading the temperature. To create new intent, go look for the "Intents" tab and click the "+" icon beside it.

  • Name the intent "device_read" and add the training phrases as shown below.

  • Then scroll down and look for the "Responses" section and "Text Response" field and add a value on the field or you may copy our example shown below. This is the default response to your "read temperature" phrases. Also on the "Fulfillment" section press the down icon beside it, then click the switch "Enable webhook call for this intent" to enable. Click the "Save" button at the top right side.

  • Test temperature reading. Type "read room temperature" on the "try it now" field on the upper right corner. The response will be "Room temperature is 31.00 in celcius". If the NodeMCU is not connected to the network or have some issue it will default to respond "Reading room temperature" only.

  • Now we will connect DialogFlow to Google Assistant. 

 

Dialog Flow to Google Assistant Setup

  • To connect DialogFlow to GoogleAssistant first go to the "Integrations" tab, and this will be displayed as shown below.

  • Next, click the "Integration Settings" near the Google Assistant Section. Add the device_control and device_read intent on the "Add intent" field. Then click the "Auto-preview changes" switch to enable, it will automatically propagate changes to the Actions Console and Assistant Simulator as shown below. Then click "MANAGE ASSISTANT APP".

  • You will be re-directed to Google Actions Console. Click the "Decide how your action is invoked" as shown below.

  • Next, provide the "Display Name" field with your desired display name, this will be used in talking to Google. In this example, we will set it as "The test app" as shown below and to talk with it is to say "Talk to The test app" to trigger the Dialogflow actions we will do later. Choose your "Google Assistant voice" type: female or male, by clicking the drop-down box. Click "Save' at the top right.

  • Now go back to the other tab "Integration Settings" as shown below.

  • Then click "TEST", wait for it as it will update your actions on Google Assistant. Then you will be directed to this webpage. 

  • Click allow on your location, then you can see the "Suggestions" section. Click "Talk to The test app".
  • Next, test the "light on", "light off", and "read room temperature" by typing it on the "Start typing or click the mic and speak" field or by clicking the mic and say out the phrases. As an example, when we say "read room temperature", the below response will be seen.

  • Now you can test it on your Google Assistant with Raspberry Pi. First, manually run the Google Assistant again by opening another terminal and input this command.
$ /home/${USER}/env/bin/python -u /home/${USER}/GassistPi/src/main.py
  • Next, say "Ok Google" then say "Talk to The test app" (this is our display name). Change "Talk to The test app" if you're using a different display name. Next, you can just say "light on", "light off", or "read room temperature" to test. To close it say "Close The test app".
  • Check out the demo video below.

 

Demo Video

 

Conclusion

While demonstrating that a light bulb can be controlled and temperature/humidity readings can be extracted using voice-commands via Google Assistant, we can extend this project to other electrical devices that can be controlled ON and OFF, as well as other sensors like smoke sensor, gas sensor, presence/motion sensor, any many more. You can create your own voice-activated Smart Home :D

But remember to always be cautious when working with high voltages as the risk of being electrocuted is always there. Check always your wirings that they are not going to short or better to ask help from people who know how to handle high voltage electricity if you are not sure.

 

Check other tutorials in this series: Integrating Google Assistant to Raspberry Pi

Raspberry Pi with ReSpeaker 4-Mic Pi Hat Integration

Integration of Youtube and Spotify with ReSpeaker 4-Mics Pi Hat on Raspberry Pi

Voice-Activated Light Control with IFFFT, Adafruit IO, NodeMCU, and Raspberry Pi

 

References

https://www.instructables.com/Quick-Start-to-Nodemcu-ESP8266-on-Arduino-IDE/

https://randomnerdtutorials.com/esp8266-web-server/

https://www.datacamp.com/community/tutorials/making-http-requests-in-python

https://miningbusinessdata.com/dialogflow-python-webhook-tutorial/

https://rominirani.com/hands-on-with-api-ai-google-assistant-writing-your-first-conversation-agent-a6a7dcdaba27

https://code.visualstudio.com/docs/python/python-tutorial

https://dashboard.ngrok.com/get-started/setup

Control lightDhtDht22DialogflowFlaskLight bulbNgrokNodemcuRespeaker 4-micsRespeaker 4-mics piTemperature sensorWebhook

Leave a comment

All comments are moderated before being published