Receiving Emails with ESP32 using IMAP Server

Hello readers, I hope you all are doing great. In our previous tutorial, we learned SMTP server and how to implement an SMTP server for sending emails with ESP32. In the previous tutorial, we also demonstrated some examples like sharing raw text, HTML text, images and text files.

So, at the transmitter end, we are using the SMTP server.

But, what about the receiver end?

At the receiver end, we use another protocol called IMAP (or Internet Message Access Protocol) and POP3 (Post office Protocol V3) for receiving the emails.

What is IMAP Protocol and How does it Work?

IMAP is an application layer (TCP/IP) protocol that is used at the receiver end to receive emails from SMTP server or mail server. IMAP follows the client/server model.

Features of IMAP protocol:

• IMAP can operate in three modes:
  1. Online mode
  2. Offline mode
  3. Disconnected mode
• IMAP uses two ports:
  1. 143 Port – Communication over this port is not secure because this is a non-encrypted IMAP port.
  2. 993 Port – Communication over this port is secure and it is used when the client (IMAP) wants to connect securely through IMAP.
• Users can access emails from a remote server without deleting the emails from the email server.
• Setting message flag – The message flag is set to help the user in keeping the track of which message has already been seen.
• Receivers can create a folder to organize mails in the hierarchy.
• Users can also download a portion of a message from the mime-multi part in case of large multimedia files.
• Organizing mail on the server.
• Check email header – user can check the email header before downloading the mail.

What is POP3?

POP3 stands for Post Office Protocol version 3.

POP3 is another protocol to receive emails. This protocol is used to access the TCP/IP mailbox. The protocol is quite popular due to its offline mail access model.

The offline access model enables the user to access the mails from the mail server on the local machine, and then delete them from the mail server.

Why IMAP is Preferred over the POP3?

• Being an offline access model, POP3 is not suitable for the ideal world. The biggest drawback of the POP3 model is that emails are permanently deleted from the server, and is it not possible to access the mails in different computers.
• In POP3 user is not allowed to manage the mails on the server whilst in IMAP protocol the user can delete, create or rename the mails on the server.
• Another drawback of the protocol is data security and safety.
• So, a new protocol is developed to overcome the drawback of the POP3 protocol. The protocol was named Internet Message Access Protocol or IMAP. Which is an online protocol and mails received via IMAP protocol can be accessed on different computers.

Though in some applications POP3 protocol is still used, but in most of the email receivers, it is preferred to use IMAP protocol over POP3 protocol.

Setting parameters for different IMAP service provider

IMAP (incoming) setting parameter for Gmail

• IMAP Server address – imap.gmail.com
• IMAP username – xyz@gmail.com
• IMAP password – password
• IMAP port – 993

IMAP (incoming) setting parameters for Yahoo

• IMAP Server address – mail.yahoo.com
• IMAP username – xyz@yahoo.com
• IMAP password – password
• IMAP port – 993
• SSL required – Yes

Similarly, other email service providers like Outlook and Hotmail, have different setting parameters.

• Components required to send and receive emails using ESP32 over SMTP server are:
• Recipient’s email address.
• Sender’s email address.
• Content to be shared over SMTP server.
• ESP mail client library.
• ESP32 module.

ESP mail client Library

To send emails with ESP32 we need to install this ESP Mail Client library. This library, make ESP32 able to send emails over SMTP server.

Step to install ESP Mail Client Library:

• To download the ESP Mail Client Library click on the link: https://github.com/mobizt/ESP-Mail-Client
• Open the Arduino IDE.
• Go to Sketch >> Include Library >> Add .ZIP Library.
• Select the downloaded ZIP file.
• Click on

Your Arduino IDE is ready to send email using ESP32.

Create a new Gmail account

It is recommended to create a new email account for sending emails using ESP32 or ESP8266 modules.

If you are using your main (personal) email account (for sending emails) with ESP and by mistake, something goes wrong in the ESP code or programming part, your email service provider can ban or disable your main (personal) email account.

In this tutorial, we are using a Gmail account.

Follow the link to create a new Gmail account: https://accounts.google.com

Access to Less Secure apps

To get access to this new Gmail account, you need to enable Allow less secure apps and this will make you able to send emails. The link is attached below:

https://myaccount.google.com/lesssecureapps?pli=1

Fig.

Arduino IDE Code, for configuring IMAP Protocol in ESP32

• As we mentioned earlier, we are using Arduino IDE as a compiler and upload into ESP32 module. To know more about Arduino IDE and how to use it, follow our previous tutorial i.e., on the ESP32 programming series.
• We have already discussed installing the ESP Mail Client Library to make ESP32 able to send emails over the SMTP server.
• This library includes multiple examples on SMTP like sending text messages, images, HTML code, text files etc. We have attached an image below for your reference.
• You can use those examples to send emails.

Fig IMAP and SMTP Example Code

Note: You can not use the exact code. Hence, you need to make some changes like replacing SSID and password with your network credentials, email address of sender and receiver, setting IMAP and SMTP parameters for respective email service providers etc, needs to be done before uploading the code. We will also describe these things during code description.

ESP32 IMAP Code

In this code, we will implement both IMAP and SMTP protocols to receive and transmit emails.

Although, we are using SMTP in this tutorial, but we have already discussed and demonstrated the implementation on SMTP protocol in our previous tutorial. So, in this tutorial we will not explain the SMTP part.

Follow our previous tutorial for detailed study of SMTP implementation in ESP32.

#include <Arduino.h>

#include <WiFi.h>

#include <ESP_Mail_Client.h>

//To use only IMAP functions, you can exclude the SMTP from compilation, see ESP_Mail_FS.h.

#define WIFI_SSID "public"

#define WIFI_PASSWORD "ESP32@123"

//-----------setting IMAP parameters------

/* The imap host name e.g. imap.gmail.com for GMail or outlook.office365.com for Outlook */

#define IMAP_HOST "imap.gmail.com"

#define IMAP_PORT 993

#define AUTHOR_EMAIL "techeesp697@gmail.com"

#define AUTHOR_PASSWORD "Tech@ESP123"

#define RECIPIENT_EMAIL "maneesha607ece@gmail.com"

//------------setting SMTP credentials----------

#define SMTP_HOST "smtp.gmail.com"

#define SMTP_PORT 465

//------IMAP Rx emails and their status

/* Callback function to get the Email reading status */

void imapCallback(IMAP_Status status);

void printAllMailboxesInfo(IMAPSession &imap);

void printSelectedMailboxInfo(SelectedFolderInfo sFolder);

void printMessages(std::vector<IMAP_MSG_Item> &msgItems, bool headerOnly);

/* Print all attachments info from the message */

void printAttacements(std::vector<IMAP_Attach_Item> &atts);

/* The IMAP Session object used for Email reading */

IMAPSession imap;

//-------SMTP sending mails and their status----

/* The SMTP Session object used for Email sending */

SMTPSession smtp;

/* Callback function to get the Email sending status */

void smtpCallback(SMTP_Status status);

 

void setup()

{

Serial.begin(115200);

#if defined(ARDUINO_ARCH_SAMD)

while (!Serial)

;

Serial.println();

Serial.println("**** Custom built WiFiNINA firmware need to be installed.****\nTo install firmware, read the instruction here, https://github.com/mobizt/ESP-Mail-Client#install-custom-built-wifinina-firmware");

#endif

Serial.println();

Serial.print("Connecting to AP");

WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

while (WiFi.status() != WL_CONNECTED)

{

Serial.print(".");

delay(200);

}

Serial.println("");

Serial.println("WiFi connected.");

Serial.println("IP address: ");

Serial.println(WiFi.localIP());

Serial.println();

/** Enable the debug via Serial port

* none debug or 0

* basic debug or 1

*

* Debug port can be changed via ESP_MAIL_DEFAULT_DEBUG_PORT in ESP_Mail_FS.h

*/

imap.debug(1);

/* Set the callback function to get the reading results */

imap.callback(imapCallback);

ESP_Mail_Session session;

session.server.host_name = IMAP_HOST;

session.server.port = IMAP_PORT;

session.login.email = AUTHOR_EMAIL;

session.login.password = AUTHOR_PASSWORD;

/* Setup the configuration for searching or fetching operation and its result */

IMAP_Config config;

/* Set seen flag */

//config.fetch.set_seen = true;

/* Search criteria */

config.search.criteria.clear();

/* Also search the unseen message */

config.search.unseen_msg = true;

/* Set the storage to save the downloaded files and attachments */

config.storage.saved_path = "/email_data";

config.storage.type = esp_mail_file_storage_type_flash;

 

config.download.header = true;

config.download.text = true;

config.download.html = true;

config.download.attachment = true;

config.download.inlineImg = true;

config.enable.html = true;

config.enable.text = true;

/* Set to enable the sort the result by message UID in the ascending order */

config.enable.recent_sort = true;

/* Set to report the download progress via the default serial port */

config.enable.download_status = true;

config.limit.search = 5;

config.limit.msg_size = 512;

config.limit.attachment_size = 1024 * 1024 * 5;

/* Connect to server with the session and config */

if (!imap.connect(&session, &config))

return;

/* {Optional} */

printAllMailboxesInfo(imap);

/* Open or select the mailbox folder to read or search the message */

if (!imap.selectFolder("INBOX"))

return;

/* {Optional} */

printSelectedMailboxInfo(imap.selectedFolder());

String uid = String(imap.getUID(imap.selectedFolder().msgCount()));

config.fetch.uid = uid;

/* Read or search the Email and close the session */

//When message was fetched or read, the /Seen flag will not set or message remained in unseen or unread status,

//as this is the purpose of library (not UI application), user can set the message status as read by set \Seen flag

//to message, see the Set_Flags.ino example.

MailClient.readMail(&imap);

/* Clear all stored data in IMAPSession object */

imap.empty();

}

void loop()

{

}

/* Callback function to get the Email reading status */

void imapCallback(IMAP_Status status)

{

/* Print the current status */

Serial.println(status.info());

/* Show the result when reading finished */

if (status.success())

{

/* Print the result */

/* Get the message list from the message list data */

IMAP_MSG_List msgList = imap.data();

printMessages(msgList.msgItems, imap.headerOnly());

/* Clear all stored data in IMAPSession object */

imap.empty();

SMTP_MSG();

}

}

void printAllMailboxesInfo(IMAPSession &imap)

{

/* Declare the folder collection class to get the list of mailbox folders */

FoldersCollection folders;

/* Get the mailbox folders */

if (imap.getFolders(folders))

{

for (size_t i = 0; i < folders.size(); i++)

{

/* Iterate each folder info using the folder info item data */

FolderInfo folderInfo = folders.info(i);

ESP_MAIL_PRINTF("%s%s%s", i == 0 ? "\nAvailable folders: " : ", ", folderInfo.name, i == folders.size() - 1 ? "\n" : "");

}

}

}

void printSelectedMailboxInfo(SelectedFolderInfo sFolder)

{

/* Show the mailbox info */

ESP_MAIL_PRINTF("\nInfo of the selected folder\nTotal Messages: %d\n", sFolder.msgCount());

ESP_MAIL_PRINTF("Predicted next UID: %d\n", sFolder.nextUID());

for (size_t i = 0; i < sFolder.flagCount(); i++)

ESP_MAIL_PRINTF("%s%s%s", i == 0 ? "Flags: " : ", ", sFolder.flag(i).c_str(), i == sFolder.flagCount() - 1 ? "\n" : "");

}

void printAttacements(std::vector<IMAP_Attach_Item> &atts)

{

ESP_MAIL_PRINTF("Attachment: %d file(s)\n****************************\n", atts.size());

for (size_t j = 0; j < atts.size(); j++)

{

IMAP_Attach_Item att = atts[j];

/** att.type can be

* esp_mail_att_type_none or 0

* esp_mail_att_type_attachment or 1

* esp_mail_att_type_inline or 2

*/

ESP_MAIL_PRINTF("%d. Filename: %s, Name: %s, Size: %d, MIME: %s, Type: %s, Creation Date: %s\n", j + 1, att.filename, att.name, att.size, att.mime, att.type == esp_mail_att_type_attachment ? "attachment" : "inline", att.creationDate);

}

Serial.println();

}

void printMessages(std::vector<IMAP_MSG_Item> &msgItems, bool headerOnly)

{

for (size_t i = 0; i < msgItems.size(); i++)

{

/* Iterate to get each message data through the message item data */

IMAP_MSG_Item msg = msgItems[i];

 

Serial.println("****************************");

ESP_MAIL_PRINTF("Number: %d\n", msg.msgNo);

ESP_MAIL_PRINTF("UID: %d\n", msg.UID);

ESP_MAIL_PRINTF("Messsage-ID: %s\n", msg.ID);

ESP_MAIL_PRINTF("Flags: %s\n", msg.flags);

//The attachment may not detect in search because the multipart/mixed

//was not found in Content-Type header field.

ESP_MAIL_PRINTF("Attachment: %s\n", msg.hasAttachment ? "yes" : "no");

if (strlen(msg.acceptLang))

ESP_MAIL_PRINTF("Accept Language: %s\n", msg.acceptLang);

if (strlen(msg.contentLang))

ESP_MAIL_PRINTF("Content Language: %s\n", msg.contentLang);

if (strlen(msg.from))

ESP_MAIL_PRINTF("From: %s\n", msg.from);

if (strlen(msg.sender))

ESP_MAIL_PRINTF("Sender: %s\n", msg.sender);

if (strlen(msg.to))

ESP_MAIL_PRINTF("To: %s\n", msg.to);

if (strlen(msg.cc))

ESP_MAIL_PRINTF("CC: %s\n", msg.cc);

if (strlen(msg.date))

{

ESP_MAIL_PRINTF("Date: %s\n", msg.date);

ESP_MAIL_PRINTF("Timestamp: %d\n", (int)MailClient.Time.getTimestamp(msg.date));

}

if (strlen(msg.subject))

ESP_MAIL_PRINTF("Subject: %s\n", msg.subject);

if (strlen(msg.reply_to))

ESP_MAIL_PRINTF("Reply-To: %s\n", msg.reply_to);

if (strlen(msg.return_path))

ESP_MAIL_PRINTF("Return-Path: %s\n", msg.return_path);

if (strlen(msg.in_reply_to))

ESP_MAIL_PRINTF("In-Reply-To: %s\n", msg.in_reply_to);

if (strlen(msg.references))

ESP_MAIL_PRINTF("References: %s\n", msg.references);

if (strlen(msg.comments))

ESP_MAIL_PRINTF("Comments: %s\n", msg.comments);

if (strlen(msg.keywords))

ESP_MAIL_PRINTF("Keywords: %s\n", msg.keywords);

/* If the result contains the message info (Fetch mode) */

if (!headerOnly)

{

if (strlen(msg.text.content))

ESP_MAIL_PRINTF("Text Message: %s\n", msg.text.content);

if (strlen(msg.text.charSet))

ESP_MAIL_PRINTF("Text Message Charset: %s\n", msg.text.charSet);

if (strlen(msg.text.transfer_encoding))

ESP_MAIL_PRINTF("Text Message Transfer Encoding: %s\n", msg.text.transfer_encoding);

if (strlen(msg.html.content))

ESP_MAIL_PRINTF("HTML Message: %s\n", msg.html.content);

if (strlen(msg.html.charSet))

ESP_MAIL_PRINTF("HTML Message Charset: %s\n", msg.html.charSet);

if (strlen(msg.html.transfer_encoding))

ESP_MAIL_PRINTF("HTML Message Transfer Encoding: %s\n\n", msg.html.transfer_encoding);

if (msg.rfc822.size() > 0)

{

ESP_MAIL_PRINTF("RFC822 Messages: %d message(s)\n****************************\n", msg.rfc822.size());

printMessages(msg.rfc822, headerOnly);

}

}

Serial.println();

}

}

void SMTP_MSG()

{

smtp.debug(1);

smtp.callback(smtpCallback);

ESP_Mail_Session session;

session.server.host_name = SMTP_HOST;

session.server.port = SMTP_PORT;

session.login.email = AUTHOR_EMAIL;

session.login.password = AUTHOR_PASSWORD;

session.login.user_domain = "";

/* Declare the message class */

SMTP_Message message;

message.sender.name = "The_Engineering_Projects";

message.sender.email = AUTHOR_EMAIL;

message.subject = "Auto_Response";

message.addRecipient("Maneesha", RECIPIENT_EMAIL);

//Send raw text message

String textMsg = "Thanks for contacting us. One of our client will contact you soon. www.theengineeringprojects";

message.text.content = textMsg.c_str();

message.text.charSet = "us-ascii";

message.text.transfer_encoding = Content_Transfer_Encoding::enc_7bit;

message.priority = esp_mail_smtp_priority::esp_mail_smtp_priority_low;

message.response.notify = esp_mail_smtp_notify_success | esp_mail_smtp_notify_failure | esp_mail_smtp_notify_delay;

/* Connect to server with the session config */

if (!smtp.connect(&session))

return;

/* Start sending Email and close the session */

if (!MailClient.sendMail(&smtp, &message))

Serial.println("Error sending Email, " + smtp.errorReason());

}

//-----------SMTP Status function-----

// Callback function to get the Email sending status

void smtpCallback(SMTP_Status status)

{

/* Print the current status */

Serial.println(status.info());

/* Print the sending result */

if (status.success())

{

Serial.println("----------------");

ESP_MAIL_PRINTF("Message sent success: %d\n", status.completedCount());

ESP_MAIL_PRINTF("Message sent failled: %d\n", status.failedCount());

Serial.println("----------------\n");

struct tm dt;

for (size_t i = 0; i < smtp.sendingResult.size(); i++)

{

/* Get the result item */

SMTP_Result result = smtp.sendingResult.getItem(i);

time_t ts = (time_t)result.timestamp;

localtime_r(&ts, &dt);

ESP_MAIL_PRINTF("Message No: %d\n", i + 1);

ESP_MAIL_PRINTF("Status: %s\n", result.completed ? "success" : "failed");

ESP_MAIL_PRINTF("Date/Time: %d/%d/%d %d:%d:%d\n", dt.tm_year + 1900, dt.tm_mon + 1, dt.tm_mday, dt.tm_hour, dt.tm_min, dt.tm_sec);

ESP_MAIL_PRINTF("Recipient: %s\n", result.recipients);

ESP_MAIL_PRINTF("Subject: %s\n", result.subject);

}

Serial.println("----------------\n");

}

}

Code Description

• The first task is adding the required header files or libraries.
• h is used to enable the Wi-Fi module and hence wireless network connectivity.
• Another library is h to enable email services for receiving and transmitting mails over IMAP and SMTP protocols respectively.

• Enter the network credentials in place of SSID and PASSWORD.

• Enter the IMAP server address and port number of the respective email service provider.
• In this code, we are using the Gmail service.

• Enter the SMTP server address and port number of respective email service provider.

Fig. SMTP server address and port number

• Enter your (source) email address and password details to receive and transmit emails.

• Insert another (destination) email address.

• imapCallback() function is used to fetch the email reading status.

• Print the list of mailbox folders.

• Print the information of the selected mail folder.

• Print all the messages from the message list.

• Print the attachment contained in the mail.

• Call the IMAP session object which is used to read emails.

• SMTPSession object is used for sending emails.

• This smtpCallback() function is used to get the email sending status.

• This function also includes printing the results like success and failure of email sent.

Setup() Code

• Initialize the serial monitor at a 115200 baud rate for debugging purposes.
• WiFi.begin() function is used to initialize the Wi-Fi module with Wi-Fi credentials used as arguments.
• The While loop will continuously run until the ESP32 is connected to the Wi-Fi network.

• If the device is connected to a local Wi-Fi network then print the details on the serial monitor.
• WiFi.localIP() function is used to fetch the IP address.
• Print the IP address on the serial monitor using println() function.

• Imap.debug(1) function is used for debugging through the serial monitor.
• Where parameter ‘1’ represents basic debug.

• imap.callback() function is set to get the reading results.

Fig.

• Declare the session configuring data.

• Some properties of session configuration and imap_configuration accept the pointer to constant char. You can also change them to a string.
• Set the session configuration which includes the server address of the IMAP service provider, port, and your email address and password.

• For searching and fetching operation results set the configuration.

• Configure the searching criteria for available information.

• Configuration for searching unseen messages.

Fig.

• Set the storage path to save the downloaded files.

Fig.

• Configure the flash for storing email files.

• Set the message (download) header which includes text, HTML message, attachments (to store the attachments you need to enable Serial Peripheral Interface Flash File System or SPIFFS).
• Follow our previous tutorial i.e., on sending emails with ESP32 over SMTP

• Enable the results for HTML and text messages which are to be stored in IMAPSession object.
• The IMAPSession object’s size is limited by config.limit.msg_size.

• Enable.recent_sort is configured to enable the feature of sorting the messages by UID in ascending order.

• download_status is configured to indicate the download progress on the serial monitor.

• limit.search is configured to limit the emails in the search result.

• The size of the message stored is limited to 512 bytes.

• The size of attachments and images that can be downloaded is limited to 1024*1024*5bytes.

• Connect to IMAP server with the set session and configurations.

• Open the selected mail folder to read the received information.

• Fetch UID from the maximum message number in mailbox and write the UID into a string type variable

• Read the message once it is fetched.

• Delete/clear all the data stored in the IMAPSession

imapCallback()

• As we mentioned earlier, imapCallback() function is used to fetch the email reading status.
• The current status is printed on the serial monitor from info() function.
• If the email is fetched successfully, print the result and message list.
• After that clear all the data stored in IMAPSession object.
• SMTP_MSG() function is called when an email is fetched.
• SMTP_MSG() function initialize the SMTP server to send an email to the destination client. This function contains all the instructions related to sending email which includes debugging, session object, message header, type of data contained in the email (raw text or HTML text), SMTP server settings, connecting to the server.
• ESP32 will automatically send an email in reply of the mail received from the client device using this SMTP_MSG function.

Fig.

Note:- We have already discussed the SMTP server in our previous tutorial that is Sending Emails with ESP32 using SMTP server. So, follow our previous tutorial for a detailed study on SMTP protocol and send an email using ESP32.

Testing

• After making the required changes in the given code like Wi-Fi credentials, email service provider’s details, email address of sender and receiver etc. upload the code into the ESP32 module.
• Make sure you have turned on the access for less secure apps as discussed earlier.
• Open the serial monitor with a 115200 baud rate.
• Send an email to the address login in ESP32 from another email.
• Make sure ESP32 is connected to the internet.
• When an email is received by ESP32, the respective results are shown below :

• Once the email is fetched completely, SMTP will be initialized and an auto-response email will be sent to the destination email address.

This concludes the tutorial. I hope you found this helpful and also hope to see you soon with a new tutorial on ESP32.