ESP-32 Cam with On Demand Image Capture

For this project I want to use an ESP-32 Cam as a security monitor and entry/exit camera. My plan is that this camera will point at a door it is monitoring; it will periodically take a picture but I also want to be able to get an immediate update. It should also be able to take a picture when someone opens and closes the door that the camera is pointing at.

We're going to be using an IoT approach and Home Assistant. The entry/exit aspect is going to be triggered by an automation from a door sensor but we will not cover the automation here. The main thing is to make a sketch and integration for on demand image capture.

Initial Testing

To make sure the camera actually works and is connected up correctly I want to run some kind of test code. From the Examples menu in the Arduino IDE, look for ESP32/Cam/CameraWebServer and modify the code for the camera type you have and your WiFi network.

My device looks similar to the AI Thinker ESP32-CAM and seems to work with that board selected. I ran the sketch and confirmed all the features seem to be working.

MQTT Topics

We need a design for the communication protocol that will be used with the camera:

Code

I'm assuming that you have already installed necesary libraries for the camera and MQTT. I took bits and pieces from various tutorials for using the ESP-32 Cam, most of which copy from the Espressif reference code on github. I added code for using AsyncMQTT libary based on my previous projects for reporting temperatures and controlling a relay.

/**
 * On demand camera sketch
 */

#include "esp_camera.h"
#include <WiFi.h>
#include <AsyncMqtt_Generic.h>
#include "esp_timer.h"
#include "img_converters.h"
#include "Arduino.h"
#include "fb_gfx.h"
#include "soc/soc.h" //disable brownout problems
#include "soc/rtc_cntl_reg.h"  //disable brownout problems
#include "esp_http_server.h"
#include "general.h"

const char *ssid = MY_SSID;
const char *password = MY_WIFI_PASSWORD;

// MQTT Topic to subscribe, that will control camera
const char *subTopic = "lh-mqtt/trigger_cam";

// MQTT Topic to publish, to show image from camera
const char *pubTopic = "lh-mqtt/cam01";

AsyncMqttClient mqttClient;

#define LED_BUILTIN 4
#define CONFIG_LED_ILLUMINATOR_ENABLED 1
#define LED_LEDC_CHANNEL 2
#define CONFIG_LED_MAX_INTENSITY 255

// This project was tested with the AI Thinker Model only
#define CAMERA_MODEL_AI_THINKER
//#define CAMERA_MODEL_M5STACK_PSRAM
//#define CAMERA_MODEL_M5STACK_WITHOUT_PSRAM

#if defined(CAMERA_MODEL_WROVER_KIT)
  #define PWDN_GPIO_NUM    -1
  #define RESET_GPIO_NUM   -1
  #define XCLK_GPIO_NUM    21
  #define SIOD_GPIO_NUM    26
  #define SIOC_GPIO_NUM    27

  #define Y9_GPIO_NUM      35
  #define Y8_GPIO_NUM      34
  #define Y7_GPIO_NUM      39
  #define Y6_GPIO_NUM      36
  #define Y5_GPIO_NUM      19
  #define Y4_GPIO_NUM      18
  #define Y3_GPIO_NUM       5
  #define Y2_GPIO_NUM       4
  #define VSYNC_GPIO_NUM   25
  #define HREF_GPIO_NUM    23
  #define PCLK_GPIO_NUM    22
#elif defined(CAMERA_MODEL_M5STACK_PSRAM)
  #define PWDN_GPIO_NUM     -1
  #define RESET_GPIO_NUM    15
  #define XCLK_GPIO_NUM     27
  #define SIOD_GPIO_NUM     25
  #define SIOC_GPIO_NUM     23

  #define Y9_GPIO_NUM       19
  #define Y8_GPIO_NUM       36
  #define Y7_GPIO_NUM       18
  #define Y6_GPIO_NUM       39
  #define Y5_GPIO_NUM        5
  #define Y4_GPIO_NUM       34
  #define Y3_GPIO_NUM       35
  #define Y2_GPIO_NUM       32
  #define VSYNC_GPIO_NUM    22
  #define HREF_GPIO_NUM     26
  #define PCLK_GPIO_NUM     21
#elif defined(CAMERA_MODEL_M5STACK_WITHOUT_PSRAM)
  #define PWDN_GPIO_NUM     -1
  #define RESET_GPIO_NUM    15
  #define XCLK_GPIO_NUM     27
  #define SIOD_GPIO_NUM     25
  #define SIOC_GPIO_NUM     23

  #define Y9_GPIO_NUM       19
  #define Y8_GPIO_NUM       36
  #define Y7_GPIO_NUM       18
  #define Y6_GPIO_NUM       39
  #define Y5_GPIO_NUM        5
  #define Y4_GPIO_NUM       34
  #define Y3_GPIO_NUM       35
  #define Y2_GPIO_NUM       17
  #define VSYNC_GPIO_NUM    22
  #define HREF_GPIO_NUM     26
  #define PCLK_GPIO_NUM     21
#elif defined(CAMERA_MODEL_AI_THINKER)
  #define PWDN_GPIO_NUM     32
  #define RESET_GPIO_NUM    -1
  #define XCLK_GPIO_NUM      0
  #define SIOD_GPIO_NUM     26
  #define SIOC_GPIO_NUM     27

  #define Y9_GPIO_NUM       35
  #define Y8_GPIO_NUM       34
  #define Y7_GPIO_NUM       39
  #define Y6_GPIO_NUM       36
  #define Y5_GPIO_NUM       21
  #define Y4_GPIO_NUM       19
  #define Y3_GPIO_NUM       18
  #define Y2_GPIO_NUM        5
  #define VSYNC_GPIO_NUM    25
  #define HREF_GPIO_NUM     23
  #define PCLK_GPIO_NUM     22
#else
  #error "Camera model not selected"
#endif

void setupWifi() {
  delay(10);
  WiFi.begin(MY_SSID, MY_WIFI_PASSWORD);
  while(WiFi.status() != WL_CONNECTED) {
    delay(500);
  }
}

void connectMqtt()
{
  Serial.println("Connecting to MQTT...");
  mqttClient.connect();
}

void onMqttConnect(bool sessionPresent)
{
  Serial.print("Connected to MQTT broker: "); Serial.print(MQTT_HOST);
  Serial.print(", port: "); Serial.println(MQTT_PORT);

  mqttClient.subscribe(subTopic, 0);
  Serial.print("Session present: "); Serial.println(sessionPresent);
  }

void onMqttSubscribe(const uint16_t &packetId, const uint8_t &qos)
{
  Serial.println("Subscribe acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
  Serial.print("  qos: ");
  Serial.println(qos);
}

void onMqttPublish(const uint16_t &packetId)
{
  Serial.println("Publish acknowledged");
  Serial.print("  packetId: "); Serial.println(packetId);
}

void onMqttMessage(char *topic, char *payload, const AsyncMqttClientMessageProperties &properties,
                   const size_t &len, const size_t &index, const size_t &total)
{
  Serial.println("Publish received.");
  Serial.print("  topic: ");  Serial.println(topic);
  Serial.print("  qos: ");    Serial.println(properties.qos);
  Serial.print("  dup: ");    Serial.println(properties.dup);
  Serial.print("  retain: "); Serial.println(properties.retain);
  Serial.print("  len: ");    Serial.println(len);
  Serial.print("  index: ");  Serial.println(index);
  Serial.print("  total: ");  Serial.println(total);

  String content = "";
  for(size_t i = 0; i < len; i++)
  {
    content.concat(payload[i]);
  }
  Serial.print(content);
  Serial.println();

  // If we subscribe to more topics we should check topic name
  if(content == "PRESS") {
    Serial.println("Capturing image");
    cameraCapture();
  }
}

void onMqttDisconnect(AsyncMqttClientDisconnectReason reason)
{
  (void) reason;

  Serial.println("Disconnected from MQTT.");
}

void setup()
{
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); // disable brownout detector

  Serial.begin(115200);
  Serial.setDebugOutput(false);

  pinMode(LED_BUILTIN, OUTPUT);

  // Camera set up

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  config.grab_mode = CAMERA_GRAB_LATEST;

  if(psramFound()){
    config.frame_size = FRAMESIZE_UXGA;
    config.jpeg_quality = 10;
    config.fb_count = 2;
  } else {
    config.frame_size = FRAMESIZE_SVGA;
    config.jpeg_quality = 12;
    config.fb_count = 1;
  }

  // Camera initialisation
  esp_err_t err = esp_camera_init(&config);
  if(err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }

  // MQTT set up
  mqttClient.onConnect(onMqttConnect);
  mqttClient.onDisconnect(onMqttDisconnect);
  mqttClient.onSubscribe(onMqttSubscribe);
  mqttClient.onPublish(onMqttPublish);
  mqttClient.onMessage(onMqttMessage);
  mqttClient.setServer(MQTT_HOST, MQTT_PORT);

  // Start network and MQTT
  setupWifi();
  connectMqtt();
}

// Capture an image
esp_err_t cameraCapture()
{
  camera_fb_t *fb           = NULL;
  esp_err_t    res          = ESP_OK;
  size_t       _jpg_buf_len = 0;
  uint8_t     *_jpg_buf     = NULL;

  digitalWrite(LED_BUILTIN, HIGH);
  delay(100); // Let the flash brighten up

  fb = esp_camera_fb_get();
  if(!fb) {
    Serial.println("Camera capture failed");
    res = ESP_FAIL;
  } else {
    if(fb->width > 400) {
      if(fb->format != PIXFORMAT_JPEG) {
        bool jpeg_converted = frame2jpg(fb, 80, &_jpg_buf, &_jpg_buf_len);
        esp_camera_fb_return(fb);
        fb = NULL;
        if(!jpeg_converted) {
          Serial.println("JPEG compression failed");
          res = ESP_FAIL;
        }
      } else {
        _jpg_buf_len = fb->len;
        _jpg_buf = fb->buf;
      }
    }
  }
  digitalWrite(LED_BUILTIN, LOW);

  // Publish image
  if(res == ESP_OK) {
    // Args: topic, qos, retain, payload, length, dup, id
    mqttClient.publish(pubTopic, 0, false, (const char *)(_jpg_buf), _jpg_buf_len);
  }

  // Clean up
  if(fb) {
    esp_camera_fb_return(fb);
   fb = NULL;
    _jpg_buf = NULL;
  } else if(_jpg_buf) {
    free(_jpg_buf);
    _jpg_buf = NULL;
  }

  return(res);
}

void loop()
{
  delay(1);
}

The file general.h contains the following and needs to be modified for your network set up:

#ifndef GENERAL_H
#define GENERAL_H

#define MY_SSID "edit me"
#define MY_WIFI_PASSWORD "edit me"

#define MQTT_HOST "edit me"        // Broker address
#define MQTT_PORT 1883             // Usual port

#endif  // GENERAL_H

Build and transfer the sketch to the device in the usual way. To test this code I opened two terminals on a machine with MQTT client tools installed, such as the mosquitto-clients package on Linux.

In one terminal I entered:
mosquitto_sub -t "lh-mqtt/cam01" > camera_test.jpg
In the other terminal I entered
mosquitto_pub -t "lh-mqtt/trigger_cam" -m "PRESS".
In the first terminal I hit Ctrl-C and then I could see a new file had been created and used scp to copy to my main computer where I could use an image viewer to verify the image.

While researching how to save the image topic I noticed that many people using Windows encountered issues with tools assuming that the payload of the message was text and then munging the text encoding. I avoided that by using Linux but if it's not working for you and you use Windows then that is the first place I would look.

Home Assistant Integration

Having installed the MQTT integration previously, adding new MQTT based devices involves editing the configuration.yaml file and getting HA to reload it.

Here is the text I added:

mqtt:
  image:
    - name: "ESP-32 Camera 01"
      image_topic: "lh-mqtt/cam01"
      content_type: image/jpg
  button:
    - name: "Cam01 Capture"
      qos: 0
      retain: false
      command_topic: "lh-mqtt/trigger_cam"
      payload_press: "PRESS"
      icon: mdi:camera-wireless

Two new cards will be available in Home Assistant:

Image Card with the camera topic info

Info for the Home Assistant Image Card showing the Camera Topic


Home Assistant dash button

Home Assistant Dashboard Camera Capture Button

Final Thoughts

My code is a little bit unrefined and will benefit from experimentation, fine tuning and dogfooding. For example, the camera capture mode and JPEG quality settings might be edited to better fit the location I will be using this camera in (the image above is only for illustration purposes). The networking code could be more robust and the Home Assistant YAML could have more details added.

The amount of effort required to get to this unpolished-but-working stage was actually very reasonable. To really make it useful we now have to determine the best time to capture an image of the person using the door because if the image it taken too soon then it will be a picture of only the door opening. It is also important to try to minimise battery usage.

Future Work

Here are some ideas to extend the project:

< Back