Bounce Processing How To Guide.
Automatic bounce processing allows the application to automatically login to your account, download your bounced emails, and then analyze them and categorize them accordingly depending on the severity of the bounce. The application sends the campaign’s ID and other details through ‘X-Mailer’ headers in the email. In order for automatic bounce processing to work you need a cron job (linux) or scheduled task (windows) running on the server to execute the application’s cron script (/admin/cron/cron.php), and you also need to set up the bounce account’s connection details for every contact list that you want bounce processing to handle.
How It Works.
When you create your contact list you define the default Bounce-To address. However, when sending a campaign you can also define a Bounce-To address. The Bounce-To address will automatically receive any emails that “bounce” off receiving mail servers. It’s typically useful to pick a Bounce-To address that is not used for any other purpose, such as
Any subscribers viewing your email headers will realize you take pride in keeping an updated, accurate subscriber list, and it will also lead to less clutter than you would experience using a normal mailbox, as bounce messages typically grow with the size of the list, meaning your mailbox could be receiving hundreds or even thousands of bounce emails per campaign send. The image below shows where to define your Bounce-To address in your Send an Email Campaign page.Your browser may not support display of this image.
You MUST fill in the bounce account’s connection details for every contact list that you want bounce processing to handle. When you’re creating a contact list the default bounce settings, which you can set default bounce details from the Settings → Bounce Settings page, are automatically filled in. You may need some extra information from your webhost or network administrator as to whether your account is POP/IMAP capable, and if there are any extra settings required to login to your bounce account’s mail server.
To see common problems and mistakes made in configuring bounce processing, please consult the FAQ at the bottom of this document.
So you’ve setup your Bounce-To address, and you’ve configured your server details. You know you have bounce emails, and you want to process them. At this point, you have two options, depending on your server configuration. If you have cron support enabled, you can schedule automated bounce processing from your Settings → Cron Settings page. This lets you adjust the frequency of bounce processing on your server. Please note that bounce processing can be very resource-intensive. It’s best to set larger intervals for bounce processing than scheduled sending or other cron tasks to lower the load upon your server. When it’s scheduled to run, IEM will access your mail server and attempt to automatically process your bounced emails. If it has any problems, it will send an email to the server administrator email address informing them of the problem and with some notes on the error experienced and possible ways to fix the problem.
If you don’t have cron enabled, or don’t wish to automatically process bounced emails, you can still process them manually. You can access this from the Contact Lists → Process Bounced Emails option on your top menu. From here, you’ll have to choose which contact list you want to process emails for, and also confirm the correct server details. Once again, if you have trouble accessing your mail account, please review the helpful hints at the bottom of this guide.
After processing your emails, subscribers will be marked appropriately, and you can access all bounce related statistics from the Statistics → Email Campaign Stats or Statistics → Contact List Stats pages. Hard bounces will make a subscriber be marked as “Bounced” by the application after one bounce, and deal with problems like “email address doesn’t exist” or “invalid domain name”. Soft bounces take 5 times to remove a subscriber from your list, and deal with more common problems like “relay problem” or “mailbox is full”.
Your server does not have the required modules install to process bounces. PHP-IMAP
If you have received this error message then you need to have your host install the PHP-IMAP module for you on your server. Interspire Email Marketer needs this module to be able to process bounced emails.
You followed the steps, read the guide, and you’re sure that you’ve done things correctly, but something isn’t working still and you can’t process your bounces. Here are some very common bounce processing errors, and the proper solutions to fix these problems.
“I can’t connect to the server in my bounce server details.” “I’m trying to connect to my bounce server but receiving a message about ‘Connection Refused’ or ‘Connection Timed Out’.”
This problem is typically caused by a port configuration/firewall issue on your web server. You’ll want to ensure that outbound connections are permitted on your server on the port you’re using, and that the destination server is accepting inbound connections. You may need to consult with your webhost to troubleshoot this issue. Here are some common email connection ports:
- POP3 – port 110
- IMAP – port 143
- Secure IMAP (IMAP4-SSL) – port 585
- IMAP4 over SSL (IMAPS) – port 993
- Secure POP3 (SSL-POP) – port 995
If your mail server is on a port other than 110 or 143, you will need to enter the port number for the "Default Bounce Server" field like this: "server_address:port_number" (ie for Gmail, it should read "pop.gmail.com:995" or "imap.gmail.com:993"). Also, if the port is different from 110 or 143, you will most likely need to enable SSL. If you are using Gmail or another provider that requires you to connect over SSL.
“When testing my bounce server details, everything works properly. However, when cron attempts to do automated bounce processing, I receive an error message about ____ in my email”.
IEM requires the use of PHP 7 and above. Often times though, many servers provide dual compatibility for both PHP 5 and PHP 7. When cron processing attempts to analyze bounces, it uses PHP’s command line interface. Many times, server administrators may have configured this CLI to run using PHP 5 rather than PHP 7, or perhaps have not given the same level of permissions to the CLI as PHP’s normal interface. You’ll need to consult your webhost and verify that the PHP Command Line Interface has the same permissions as normal PHP, and that it runs using PHP 7.
“I receive a message about installing the ‘PHP-IMAP’ module when I attempt to process bounces. What is this module and why do I need it installed?”
The PHP-IMAP module is a special code library that adds upon the normal functionality of PHP, and enables extra functions such as the ability to access IMAP and POP3 email accounts to download, view, and delete emails from remote email accounts. Some servers don’t install this module by default, and you need to consult your webhost directly and ask them to re-compile PHP with support for the PHP-IMAP module.
“My bounce email inbox seems to have more bounce messages, but bounce processing does not remove these messages. Why not?”
The application takes a number of steps when verifying whether or not an email is a bounce email. Firstly, it analyzes the subject of the message for known bounce email subjects such as “Deliverability Failure” or “Bounce Notification”. It must match one of these subjects before the application proceeds to process the email further. Many times these bounce messages can fail if the subject is a reply to the original message, which is the way some receiving mail servers have configured their bounce messages to do.
For example, a bounce from the email campaign with subject “X-Mas Specials!!!” could return as “Re: X-Mas Specials!!!”, and the application does not properly categorize it as a bounce. To fix those issues, just follow these steps:
- First you should enable bounce debugging so you can see which emails are getting ignored. To do that open /admin/functions/api/bounce.php and change line 40 to true, so that it looks like:
var $Debug = true;
- With debugging enabled go to Contacts Lists -> Processed Bounced Emails, select your list and click next, select process bounces manually and click test connection and continue. Click Find bounces on the next page.
- As it goes through the bounces it will generate a log in the /admin/temp/ directory in your installation called bounce.debug.log. Open that log and look for emails that were ignored.
As an example, one entry in the log might look like "Line 1013; Subject ‘test’ does not match pattern". To fix that you will need to open your bounce rules and then add ‘test’ to the array of subjects.
Another example is if the subject was recognized but the email body was not, such as this entry:
Line 774; processing body: In connection with my maternity leave until mid-May I mail my check and respond less frequently. It could be a while before you answer.
Line 819; Looking for RFC 3464 & RFC 3463 pattern Line 879; RFC pattern not found, continuing with rules
To fix that you will need to open your bounce rules and add a string from the body to a rule, like:
$GLOBALS['BOUNCE_RULES']['delete']['delete'] = ' maternity leave';
I chose "delete" because it seems like the subscriber will still be receiving the email, so the bounce message can be deleted/ignored.
If you are unable to resolve this issue with the above steps, please open a ticket from your client area and we can help you further.
“There are bounce emails in my bounce email inbox, but they have subjects that should have been caught by the application. What are some possible causes?”
If after reading the previous ticket, you’re positive there should be a subject match for your bounce messages, there may be some other causes. To identify which subscriber and which list the bounce message deals with, the application attaches special headers to the email so it can identify the subscriber later. Sometimes, mail servers are not configured properly and they strip these extra headers, and full bounce messages are not processed correctly as a result. Enable full headers (From Thunderbird, go to View → Headers → All. From Outlook Express, right click your message, and go to Properties → Details. If you are missing the “X-Mailer-SID” and “X-Mailer-LID” headers, you’ll need to change settings on your mail server to not strip these extra headers, and this should fix your issue.
“I have the right subject, and I have the right headers on my bounce messages. Is there any other reason the application isn’t catching my bounce messages?”
Sometimes, the bounce rules simply can’t positively identify your bounce messages properly. Mail servers use many different message formats and obscure error codes, and it’s hard to keep up with the times. Luckily, Email Marketer gives you the possibility to edit your own bounce rules to add to the default rules defined in Email Marketer. You can add these rules into the /admin/resources/userbouncerules.php file, following the format used in the /admin/resources/bounce_rules.php file. This will enable you to add onto the functionality of Email Marketer and ensure you can properly categorize every bounce message you ever receive.
“I don’t seem to have any more bounce emails in my bounce email inbox, but the inbox still has other unrelated messages (autoresponders, out-of-office replies, etc.). How can I wipe these emails from my inbox?”
There is a setting that lets you wipe ALL emails from your inbox after bounce processing is completed. This option is located at the bottom of the bounce server details box, and will wipe any other emails also after bounce processing completes. Please beware though, as this should not be used with manual bounce processing if the email inbox is used for multiple contact lists, as manual bounce processing can only process one contact list at once and true bounce messages for other lists might be wiped before bounce processing properly analyzes them.
“My problem isn’t really described by the above issues, or these don’t seem to fix my problem. What should I do?!”
If you’ve followed this guide from end to end, you’re probably pretty confident that you understand every aspect of Interspire’s bounce processing tool. Unfortunately, sometimes you can’t fit every problem into an easy solution. If you are positive you have followed this guide and still can’t get bounce processing working, please open up a ticket in the client area. Our support engineers will review your installation, your configuration settings, and attempt to identify the problem.
Can I use a Gmail account to handle bounced emails?
To process bounces against a Gmail account, set your bounce settings as follows:
- Bounce Server: imap.gmail.com:993 OR pop.gmail.com:995
- Bounce Username: (either ‘user’ or ‘[email protected]’ will work)
- Bounce Username:
- Check Yes, this is an IMAP account only if you are using imap.gmail.com:993.
- Check Yes, use extra mail settings and Other, then for Other put /ssl/novalidate-cert
You also will need to have activated IMAP access OR activated POP access in your Gmail account.
If this does not work and you get the error "Invalid remote specification" then your php_imap module has not been compiled with SSL support. You will need to ask your web host to recompile PHP and add the –with-imap-ssl option to the configure command.
If Gmail marks the bounces as spam, simply mark them as safe inside the Gmail Junk folder.
When trying to process bouces, I get Certificate failure error
If you receive an error similar to the message below, you need to tick the "Do not validate certificate" option under "Extra Mail Settings"
Certificate failure for xxxxxxxxxxxx: self signed certificate: /C=--/ST=xxxxxx/L=xxxxxx/O=xxxxxxx/OU=xxxxxx/CN=xxxxxxxxx/[email protected] Please check the details and try again.
After processing bounced emails there are still emails in the email account. Isn’t Interspire Email Marketer supposed to delete these emails?
Interspire Email Marketer will delete any bounced emails inside the bounce email account that it recognizes as either soft or hard bounces. If it doesn’t recognize an email as a bounce then it will ignore it.
As of version 5.5 you can tell Interspire Email Marketer to delete ALL emails in your bounce inbox when bounced emails are processed. To do that, follow these steps when setting up your bounce processing details (either from the Settings page under the "Bounce Account Details" section on the "Email Settings" tab, or when specifying bounce server details when adding/editing a contact list):
- Tick the I understand that bounced emails will be removed from the inbox I am using to manage them option. A new checkbox will appear
- Tick the Remove all emails from the inbox, not just bounce emails checkbox to permanently delete all emails in the inbox when bounce processing is finished.
Interspire Email Marketer uses a bounce rules file to detect and process bounced emails. You can modify and add your own bounce rules by editing this file (please note that we do not provide support for modified bounce rule files):
To add a new rule follow these simple steps:
- Find a consistent message in the bounced email – for example User Does Not Exist
- Do NOT include I.P. addresses or server names in the rule because if you get a similar bounce from another server, it will not pick up properly.
- Next find the section you want to add it to, for example:
- Copy a line like the one you want to modify, and change the rule:
$GLOBALS['BOUNCE_RULES']['hard']['emaildoesntexist'] = 'your new rule goes here';
If I run my cron.php file very frequently will I run into problems. Why?
If you’re going to run cron.php very often, you’ll need to set up a separate cron job for bounce processing.
To do that, edit the admin/cron/cron.php file and comment out the ‘bounce.php’ line:
# require($mydir . '/bounce.php');
and set up a separate cron job to run ‘admin/cron/bounce.php’ once an hour (or even once every 2-4 hours, doesn’t matter).
This is to reduce the load on your mail server by not checking for bounces every time that you send email campaigns or autoresponders.
Please NOTE that this method is intended for SendStudio older than 1.3… In the newer version, you can simply create one CRON entry, and specify the interval of "Send", "Autoresponder", and "Bounce Processing" separately in your SendStudio "Settings" page.
Should I use a POP3 or IMAP account for bounce processing?
It doesn’t matter whether you use a POP3 or IMAP account as your bounce email, Interspire Email Marketer will work with either.
The differences will arise when you start to optimize your mail server or if one mail server has more load placed on it.
How do I get the email list of invalid/bounced emails from aparticular contact list, so we can remove those users from that list.
When you send an email campaign out to a contact list you can set the bounce address that you want your bounced emails to be sent to.
Once your bounces come back to that address you can use the feature ‘Contact Lists->Process Bounced Emails’ to have Interspire Email Marketer connect to that mail account and search through the bounced emails. It will then change the status of your contacts that have hard bounced, or soft bounced 5 times, to Bounced.
From here you will no longer send to these email addresses. Interspire Email Marketer does not delete the addresses to make it easier for you to know what addresses are invalid if you need to re import any.
If you would like to see your bounced emails you can filter your contact lists by ‘Status->Bounced’.
I’ve processed bounces but there are still bounce emails on my server that I know should have been removed according to my rules. Why?
If you send your bounced emails to the one account, say [email protected], from all contact lists and then you process bounced emails for ‘contact list 1’ you will still have emails in your [email protected] in box from ‘contact list 2’.
You will need to process bounces for each and every contact list that you have to remove all the bounced emails.
If you want Interspire Email Marketer to automate this you will need to store the bounce account details in the Contact List creation process.
I have saved my bounce information and sent an email campaign. How do I process the bounces so that I wont send to bad email addresses?
If you have set bounce information for your email campaign sends you can then go to the ‘Contact Lists’ menu and select ‘Process Bounced Emails’ .
From here you will be asked to select a contact list that you wish to process.
Simply follow the step by step process to remove any bounced emails from your list.
Should I set up different bounce accounts for each contact list?
There is no real difference in how you set up your bounces. It is merely personal preference.
If you wish to keep your bounces completly separate then you can set up different bounce accounts. This can be helpful if you have different clients using the application so that the bounces go to @domain.com for the domain.com client and to example.com for the example.com client.
If you do not mind your bounces all arriving at the same location then you can set up the one account and send all emails to this account.
If you expect to have a lot of bounces, say you are sending to 3 lists of 500,000 contacts then it could be an idea to keep the bounce accounts separate so that they do not fill up and cause problems.
How do I change the number of times a soft bounce can occur before it becomes a hard bounce ?
Edit the admin/functions/api/subscribers.php file and search for:
var $softbounce_count = 5;
You just need to change the 5 to the number of times emails to a contact can bounce for a soft reason (e.g. the user is over their disk quota) before it becomes a hard bounce.
I am receiving a lot of spam in my bounce account. What can I do to stop this?
This generally happens when you have a very publicly visible email account for your bounces, or if it is something that can be considered common such as ‘[email protected]’ .
You can change this address as often as you like so if you start to receive too much spam into your mail box simply start with a fresh one.
Why can’t I connect to an external IMAP server.
If you are hosted on BlueHost it appears that they do not allow you to connect to IMAP servers outside of their network. You will have to take this matter up with them.
How do I set up automatic bounce processing?
If you have set up your cron to run the ‘admin/cron/cron.php’ file then you are already automatically processing your bounced emails.
You now only need to set up the one cron job on your server. Interspire Email Marketer will take note of the last time the cron file was run and if it has not been longer then the interval you have set for either the ‘Scheduled Sending’, ‘Autoresponders’ and ‘Bounce Processing’ then Interspire Email Marketer will step over that process and move onto the next.
What is the difference between ‘List Bounce Email’ and ‘Bounce Account Details’ ?
The difference between the ‘List Bounce Email’ and the ‘Bounce Account Details’ is this:
List Bounce Email is the email address that all bounced emails are sent to. That is when an email address bounces and sends a ‘bounce email’ back to your server that email will go to the bounce email address that you set up.
Bounce Account Details are the details that Interspire Email Marketer uses to be able to connect to the mail server and in particular the email account that you set up as the List Bounce Email. This allows Interspire Email Marketer to view the emails and determine soft, hard and other bounces and change your contact status accordingly.
How do I save the bounce information so I don’t need to enter it in all the time?
To save your bounce account information on the "process bounced emails" page, edit your contact list and enter in the bounce information there. The next time you process bounced emails, the information will be stored.
Your bounce information includes your bounce mail server, user name and password. You may also need certain extra mail settings if Interspire Email Marketer is unable to connect to the account.
Once you have saved your bounce information in the contact lists, if you have your cron jobs set up to process bounces then your bounce processing will be handled automatically.
Where can I set up my bounce account information so I can process bounced emails?
There are two places where you can set up bounce processing.
Firstly you can set the bounce account information up in the Settings page.
Secondly you can add separate bounce account details when you create or edit a contact list.
If you save these details with the contact list then Interspire Email Marketer can process bounces automatically if you have your cron set up.
If you do not have cron set up then you can still process bounces manually by selecting the ‘Contact Lists->Process Bounced Emails’ within Interspire Email Marketer. This will let you process the bounces for each contact list.
Why do my bounce account details change from the default ones that I have set up in the Settings page?
If you set the default bounce account in the settings page then when you create a contact list the bounce details will be pre-filled with the details that you have defined in the settings page.
If you then modify the bounce details in the contact list that contact list will use the new details rather then the ones set in the Settings page.
You can then modify these details again at the time of sending so that you can use a different bounce account/settings for any particular send.