Category Archives: PHPList

PHPList config.php Tweaks

PHPList, Web Hosting, , , , , ,

PHPList Config File

PHPList works great for large email lists but, I found out that the configuration file can be tweaked to make certain aspects of it work even better.

The file to modify is config.php located under the PHPList/config folder which should be underneath the public_html folder on most installations.

 Batch Processing Tweak

I have found that I have to frequently hit process queue on PHPList to get the entire email list to process. Recently I found that changing the following settings has made a big improvement in it’s performance. My service provider limits emails to 500 per hour so I set the MAILQUEUE_BATCH_SIZE at 480 and the batch period at 3600. Now it will send out the messages as fast as possible up to the limit of 480. It seems to perform better if it can “get the messages out” fairly quickly. It will send out a bunch with a 2 second spacing that I have set ( see below ) and it seems to get them out most of the time before what ever timing was running out that was making me have hit process queue again.

 

## if you send the queue using commandline, you can set it to something that complies with the
## limits of your ISP, eg 300 messages an hour would be 
define("MAILQUEUE_BATCH_SIZE",480);
define("MAILQUEUE_BATCH_PERIOD",3600);

Make sure the MAILQUEUE_BATCH_SIZE and MAILQUEUE_BATCH_PERIOD is commented out, it might conflict with what is done above….

## if you send the queue using your browser, you may want to consider settings like this
## which will send 10 messages and then reload the browser to send the next 10. However, this
## will not restrict the sending to any limits, so there's a good chance you will
## go over the limits of your ISP
#define("MAILQUEUE_BATCH_SIZE",10);
#define("MAILQUEUE_BATCH_PERIOD",1);

So I don’t overload the server by trying to send out messages really fast I have set a slight delay…

# to avoid overloading the server that sends your email, you can add a little delay
# between messages that will spread the load of sending
# you will need to find a good value for your own server
# value is in seconds, and you can use fractions, eg "0.5" is half a second
# (or you can play with the autothrottle below)
define('MAILQUEUE_THROTTLE',2);

Queue Processing Reporting

It is also nice to see the results and know that the message actually went out. So I have it set to report this result…

# after every run of the queue to send out messages, phpList will send a summary to the
 # admin address. If you want to stop this, set this to false or 0
 define('SEND_QUEUE_PROCESSING_REPORT',true);

 

Send Only One Test Email

If you send a lot of test emails to yourself and it is a pain to get a text and HTML every time you can restrict it to just sending one. The one it will send is based on the way the user preferences are set up.

# test emails
 # if you send a test email, phplist will by default send you two emails, one in HTML format
 # and the other in Text format. If you set this to 1, you can override this behaviour
 # and only have a test email sent to you that matches the user record of the user that the
 # test emails are sent to
 define('SEND_ONE_TESTMAIL',1);

 Click Tracking

Click tracking is a method of tracking where email recipients go based on the links in the body of the email. You can track a decent amount of information on who goes to what links and even the timing of it. I have found this feature very useful as it can give an idea of how well your email is penetrating the market. To use this feature you have to use the [CLICKTRACK] directive in the message body or in a template.

# Click tracking
# If you set this to 1, all links in your emails will be converted to links that
# go via phplist. This will make sure that clicks are tracked. This is experimental and
# all your findings when using this feature should be reported to mantis
# for now it's off by default until we think it works correctly
define('CLICKTRACK',1);

# Click track, list detail
# if you enable this, you will get some extra statistics about unique users who have clicked the
# links in your messages, and the breakdown between clicks from text or html messages.
# However, this will slow down the process to view the statistics, so it is
# recommended to leave it off, but if you're very curious, you can enable it
define('CLICKTRACK_SHOWDETAIL',1);

 

Attachments

It is worth turning on the attachments feature in PHPList to be able to add attachments to the email. Turning it on creates a new tab in the PHPList menu while you are creating a message that allows you to add an attachment.

# attachments is a new feature and is currently still experimental
# set this to 1 if you want to try it
# caution, message may become very large. it is generally more
# acceptable to send a URL for download to users
# if you try it, it will be appreciated to give feedback to the
# users mailinglist, so we can learn whether it is working ok
# using attachments requires PHP 4.1.0 and up
define("ALLOW_ATTACHMENTS",1);

# if you use the above, how many would you want to add per message (max)
# You can leave this 1, even if you want to attach more files, because
# you will be able to add them sequentially
define("NUMATTACHMENTS",1);

# when using attachments you can upload them to the server
# if you want to use attachments from the local filesystem (server) set this to 1
# filesystem attachments are attached at real send time of the message, not at
# the time of creating the message
define("FILESYSTEM_ATTACHMENTS",1);

Bounce Processing

I have set the purge and purge unprocessed to 1 that way the mailbox doesn’t fill up. I want it to empty when the bounces are processed every month. I set the unsubscribe threshold to 15 instead of the normal 5. This way it really gives a lot of chances for the person to straighten out their email before it stops sending to them. It is no big deal to let it send mail that will bounce back for a while, in my opinion. But eventually I want the mailing list cleaned of deadwood, even if it takes a year it is OK.

# set this to 0 if you want to keep your messages in the mailbox. this is potentially
 # a problem, because bounces will be counted multiple times, so only do this if you are
 # testing things.
 $bounce_mailbox_purge = 1;
# set this to 0 if you want to keep unprocessed messages in the mailbox. Unprocessed
 # messages are messages that could not be matched with a user in the system
 # messages are still downloaded into PHPlist, so it is safe to delete them from
 # the mailbox and view them in PHPlist
 $bounce_mailbox_purge_unprocessed = 1;
# how many bounces in a row need to have occurred for a user to be marked unconfirmed
 $bounce_unsubscribe_threshold = 15;

Wishlist

I would like to find a way to increase the timeout for PHPList when editing a message. The timeout that I am referring to is the one that occurs if you don’t save the draft message periodically and one hour goes by. Then you wind up losing your work as the PHPList makes you resign in. So if you forget to save or get interrupted and come back later, a lot of work can be lost accidentally. Even if it had a feature such as WordPress where it periodically saved a draft, that would be a big help.

Automating PHPList Process Queue

PHPList, Web Hosting, , , , , ,

Frequently when using the non command line version of PHPList which I have loaded on my service providers server, I have to hit Process Queue multiple times for a message. I have read a lot of posts online complaining this issue. I have done some troubleshooting of it myself by turning on the verbose feature in PHPList and I am still at a loss as to what it gets stuck on and then stops processing the queue.

PHPList works great, I love it for all that it does and does well. My one complaint is the Process Queue issue and from reading a bunch of posts online I have successfully patched together a solution.

This code can be executed on the server where PHPList lives if they will let you run a wget command from a CRON entry, some won’t. In my case I opted to simply run the command from a Raspberry Pi server that I have that runs 24/7. It is simple a looping script that does the equivalent of hitting the “Process Queue” button on PHPList at regularly timed intervals and detected when the queue has been processed. I have seen several examples of using wget to be fired off every 5 minutes to do this, but that means it gets run every 5 minutes, all day, every day. Why not run something that can be scheduled for a time and then runs to completion which this script does.
This script also produces a log file that can be copied to a location viewable via the server so you can double check to see if all ran OK. I see this mostly as a debug feature and it could potentially be turned off along with some of the “echos” of text. Except for the one in the else-if, it is an error to have an empty else-if in a bash script.

PHPList Automatic Process Queue Script

#!/bin/bash
 # Controller Script for PHPlist Process Queue
 # Erick Clasen blog.oils-of-life.com 05/16/2015
# User defined variables
 # No trailing / on DIR!
 DIR=/home/erick/php-proc
 LOG=php-pq.log
 # Time to sleep between loop iterations.
 TIME=3m
 # Max number of loops, so the script breaks out if for some reason it fails to complete.
 LOOPS=100
 # Publish a copy of the log to the following directory
 PUBDIR=/home/erick/public_html
# wget having trouble with some options so simply switch to the dir we want it to work in.
 cd $DIR
x=1
 echo Start------------- >> $LOG
# While loop, breaks out if done or at LOOPS as a guard against #infinite loop.
 while [ $x -le $LOOPS ]; do
# Timestamp in DDMMYYYY_HHMM-SS (UTC)
 timestamp=$(date -u +"%d%m%Y_%H%M-%S")
echo $x $timestamp
 echo $x $timestamp >> $LOG
x=$(( $x + 1 ))
# Use wget to run process queue with login and password provided
 wget 'your_url_here.com/phpList/admin/?page=processqueue&login=phplist&password=your_password_here'
# Test to see if PHPlist process queue has completed.
 if grep -q "Finished, Nothing to do" $DIR/index.*
 then
 # Exit from Loop with a break, mark log,remove index files that wget fetched, break out.
 echo All Done! Breaking Out
 echo All Done!--------- >> $LOG
 echo ------------------ >> $LOG
 # Publish Log, Optional
 cp $LOG $PUBDIR/$LOG
 rm $DIR/index.*
 break
 else
 # Nothing Happens
 echo Keep Running...
 fi
# Sleep for TIME
 echo ------- Sleeping Process --------
sleep $TIME;
done;

PHPList Automated Process Bounces Script

Process Bounces can be run from a script as well…

#!/bin/bash
 # Controller Script for PHPlist Process Bounces
 # Erick Clasen blog.oils-of-life.com 05/16/2015
# User defined variables
 # No trailing / on DIR!
 DIR=/home/erick/php-proc
 LOG=php-pq.log
# Publish a copy of the log to the following directory
 PUBDIR=/home/erick/public_html
# wget having trouble with some options so simply switch to the dir we want it $
 cd $DIR
# Use wget to run process queue with login and password provided
 wget 'your_url_here.com/phpList/admin/?page=processbounces&login=phplist&password=your_password_here'
 echo ---------------------------- >> $LOG
 echo BOUNCES HAVE BEEN PROCESSED! >> $LOG
 echo ---------------------------- >> $LOG
 # Publish Log, Optional
 cp $LOG $PUBDIR/$LOG
 rm $DIR/index.*

CRON entry to run these scripts

These entries run Process Queue every Monday, Wednesday and Friday (1,3,5) at 4:15AM and Process Bounces at 12:15AM on the first day of the month. Nothing magic about those settings, you can set them any way you wish. I figure starting the process in the wee hours of the morning will allow people to receive the email by the time that they wake and first check their mail. I set the time to 15 after so it won’t run at the same time as some processes that I have set to run on the hour. Sending mail on weekdays keeps people from having to deal with more email than they might like on the weekends.

15 04 * * 1,3,5 /home/erick/php-proc/php-pq.sh
 15 00 1 * * /home/erick/php-proc/php-pb.sh

PHPList Automatic Process Queue Override

If you must make PHPList process the queue now and don’t want to wait for the CRON task. Two options are available, do it the via the normal way from the PHPList admin website or you can use something as simple as a CGI script to force the php-pq.sh script to execute. Below is an example of a dirt simple CGI script. It prints out the tail of the php-pq.log and runs the php-pq.sh script. Php-pq.log is the log of the previous automatic processing event and then goes ahead and fires off the Process Queue Override by running the script. Basically the act of loading the page makes it run. I keep my cgi-bin directory locked down so no one outside of a few IP addresses can get to the files there. But it would be fairly harmless if someone did, the worst that happens is that the queue gets processed.

CGI Code for Automatic Process Queue Override

#!/bin/bash
 echo "Content-type: text/html"
 echo ""
 echo "PHPList Automatic Process Queue Override"
 echo ""
 echo "
PHPList Automatic Process Queue Override from host $(hostname -s &)
"
 echo ""
 echo "
 $(tail /home/erick/php-proc/php-pq.log &)
"
 echo ""
 echo "
Information generated on $(date &)
"
 echo ""
 echo "Processing..."
echo "
 $(/home/erick/php-proc/php-pq.sh &)
"
echo "
"

 

PHP Config File Tweaks

I am working on taking some notes on some changes I have made to the PHPList config.php file and will be posting on this in August 2015.