Bounced emails parsing

using Sendy and EasyESP

Due to the ‘Sendy’ protocol imperfection, there is no possibility for a user who uses his SMTP to know how many messages are bounced and from which email addresses, in order to manage the database in accordance with the results obtained.

However we can resolve this information from logs of Postfix smtp server that is logging the entire course of message delivery process.

The following scripts runs on the same server where ‘EasyEsp’ and ‘Sendy’ are installed, and they are parsing postfix log files to create CSV files with different categories of emails.

So, let’s see how this can be done.

1. Before sending a new campaign to a list, if a previous campaign still sends delayed messages, you can check and stop them.

To check, run:

postqueue -p|wc -l

To stop, run:

postsuper -d ALL

And it is very IMPORTANT to empty the log file, so that it will only be rewritten with the necessary data.

To do this, run:

cp /dev/null /var/log/mail.log

Now you can send the new campaign with ‘Sendy’.

2. From this moment on, you can check what happens with the messages sent.
While the campaign is sending, you get results up until then; or when it is finished, the results are complete.

Let’s start by running:
(This command line creates a file ‘dsn.txt’ that will be rewritten each time with the new data.)

grep "" /var/log/mail.log | grep -o " dsn=[0-9].[0-9].[0-9]" | sort | uniq -c > dsn.txt
cat dsn.txt

You get the number of emails sent and their status codes:

The ‘Status Codes’ are explained here: Enhanced Status Codes for Delivery – RFC 1893

EX:

dsn=2. sent
dsn=4. deferred
dsn=5. bounced

dsn=5.1.1 User unknown
dsn=5.4.4 Host not found
dsn=4.0.0, Temporary failure, please try again later.

The formula for viewing emails by status is the following:
We only have to change x with 5 to view all bounced emails. Then run:

grep " dsn=x." /var/log/mail.log | grep -o -P " to=<(.+?)>" | sort | uniq -c

If we want to see also why they bounced, we will use the whole ‘Status Codes’:
Status Codes dsn=5.2.2, for example, it means ‘user quota exceeded’ or ‘Mailbox full’. Then run:

grep " dsn=x.x.x" /var/log/mail.log | grep -o -P " to=<(.+?)>" | sort | uniq -c

  area to be used once only:


                                                                                            * in this area the text will be deleted if you refresh this page !

 

3. The two previous cases can be used for a short list or for testing.
But if the list is big, better to create files for each status in the ‘status’ folder.
Then extract only the email addresses for each status in the files in an ’emails’ folder.
So edit and change four times x.x.x with 5 or 5.2.2, for example, and run:

mkdir -p status
mkdir -p emails
grep " dsn=x.x.x" /var/log/mail.log | grep -o -P " to=<(.+?)>" | sort | uniq -c > status/x.x.x.csv
grep -E -o "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b" status/x.x.x.csv > emails/emails-x.x.x.csv

Note: Pay attention to the backslashes, must be like this:

  area to be used once only:


                                                                                            * in this area the text will be deleted if you refresh this page !

 

4. Download on your Desktop the ’emails’ folder.
Go at ‘https://sendy.withyourdomain.com/login’.
Then: ‘View all lists’ -> click to List ‘name’ -> ‘Mass unsubscribe’ (or ‘Delete subscribers’) -> ‘Select file’ -> ‘Import’
In this way we have unsubscribed (or delete) the bounced or deferred emails, the one we want.