Hello!
Previously we gave an overview of our new WordPress plugin to integrate your Jenkins build process within the WordPress administration area.
The previous post gave an overview of how to set up Jenkins as well as how to set up the WordPress plugin. Where we stopped short is actually integrating the build script within Jenkins that the WordPress plugin triggers!
Why you need a build script for Jenkins to trigger when pushing WordPress from staging to production
The answer to this question may be obvious to some, but not all. The use cases may vary from scenario to scenario, but the ideal solution is to implement a solution that cleanly copies all files as well as the database from your staging or development environment over to your production or live environment.
Where it gets slightly complicated is things like your site URL may change as well as your database configuration settings in wp-config.php. Additionally you might want to display some sort of “maintenance” message on your live site while the push is in progress. I’ll try to go over how all of these requirements are addressed step-by-step in this post. The full script will be available at the bottom of this post.
Get user arguments and sanitize script input
Our push script only takes one argument : the site domain. This allows us to configure this script to be used across multiple WordPress sites. We have a configuration file that is sourced within the script where all the variables and settings needed to accomplish the push process are stored (safely).
#check command input
if [ -z "$1" ];
then
echo "JENKINS WP PUSH"
echo "---------------"
echo ""
echo "Usage : ./jenkins-wordpress.sh sitename.com"
echo ""
exit 0
fi
# Declare variables
currentdate=`date "+%Y-%m-%d"`
scriptpath="/usr/local/bin/jenkins"
# Command arguments
site_name=`echo "$1" | awk -F "." '{printf "%s\n" ,$1}' | sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' | sed 's/-/_/g' | awk -F. '{str="";if ( length($1) > 16 ) str="";print substr($1,0,15)str""$2}'`
You can see there are some global variables being declared like the date and path of the script. The site_name is the variable that takes the first argument (domain) and sanitizes it so its all lowercase.
Have the script load configuration variables for each push
This is important because we dont want to hard code settings, passwords and connection strings in the script itself. We also want to make the script dynamic and able to work with multiple websites, not just one.
# Get configuration variables
source ${scriptpath}/config/${site_name}.conf
In shell scripting, all you need to do is “source” a config file and it will read all the VAR=”WHATEVER” declarations separately. The config file declares the staging database credentials, staging hostname, staging connection user and source directory. Similar declarations are made for production
# staging staging_db_name="staging_db_name" staging_db_user="staging_db_user" staging_db_password="staging_db_pw" staging_db_host="localhost" staging_host="staging.site.com" staging_user="ssh_user" source_dir="/var/www/staging.site.com/public_html" # production prod_db_name="prod_db_name" prod_db_user="prod_db_user" prod_db_password="prod_db_pw" prod_db_host="localhost" destination_dir="/data/www/site.com/public_html" destination_user="ssh_user" destination_host="site.com"
Display a maintenance message when pushing changes to your WordPress site
Even though some changes may be only file based changes, our script clones the staging WordPress site over to production every time a push is triggered. We could break down the push types by adding extra arguments to the script logic, but a clean slate approach for every push is the easiest approach to start with.
The site is down temporarily for maintenance. We will be back online shortly.
We want a flat file maintenance HTML message to activate before anything actually happens. Then we want this to be turned off after all the work is complete. The reason why we wont be using WordPress maintenance message plugins is simply because we are copying the database during the push process so any plugin that displays a message within WordPress will break during that process, rendering it pointless.
In Nginx we have an if-condition in the live site configuration to redirect all requests to the maintenance html file if the file maintenance_on.html exists :
location / {
if (-f $document_root/maintenance_on.html) {
return 503;
}
index index.html index.htm index.php;
}
error_page 503 @maintenance;
location @maintenance {
rewrite ^(.*)$ /maintenance_on.html break;
}
In the main “/” declaration above, we have an if-condition that states, if the file maintenance_on.html exists, return a 503 error. We then declare 503 errors to be rewritten to the maintenance_on.html page for all requests.
This is possible to accomplish in apache as well, however I won’t go into it here. Having this in place for your push script can be applied to any site, not just WordPress. This allows your script to “flip the switch” on the maintenance message instantly and easily :
# Enable Maintenance Mode
echo "Enabling maintenance mode.."
ssh -l $destination_user $destination_host \
"cd $destination_dir;\
mv maintenance_off.html maintenance_on.html"
Copy files from staging WordPress site to production
Now that the maintenance message is up, we can do whatever we want! First we should copy over all the files. We use the rsync command to copy files from staging to production. This allows us to make a differential copy, delete files that exist on production but not staging and exclude certain key files that we don’t want to overwrite :
# Transfer Files
echo "Transferring files from staging to production.."
ssh -l $staging_user $staging_host \
"cd $source_dir;\
/usr/bin/rsync -rlptDu --exclude='wp-config.php' --exclude='.htaccess' --exclude='shift8-jenkins' --exclude='maintenance_on.html' --exclude='maintenance_off.html' --delete ${source_dir}\ ${destination_user}@${destination_host}:${destination_dir}"
You can see how dynamic this command is! Most of the variables are pulled from the config file. As long as you have ssh key based authentication set up correctly on all the servers involved, the above command should work. The script assumes that the staging server is on a different server than the build script. This is the most likely scenario because typically you wouldn’t expect the Jenkins installation to live on the same server as the staging server. If this is not the case then you would have to adjust the script to accommodate the connection hierarchy. In the above scenario, we ssh to staging and initiate an rsync from staging to production. This means you need access to staging from jenkins and to production from staging. Hopefully thats not too confusing!
Copy the database from staging to production in WordPress
This is the most important part! In our script example below, we will copy the staging db to a temporary file on the jenkins server. Then we will copy the database from the temporary file to production :
# Transfer Database to temp file
echo "Dumping staging database to temp file.."
ssh -l $staging_user $staging_host bash -c "'
/usr/bin/mysqldump -u $staging_db_user --password=\"$staging_db_password\" -h $staging_db_host $staging_db_name
'" > ${scriptpath}/sqltmp/sqltmp.sql
# Transfer Database to production
echo "Transferring staging database to production.."
cat ${scriptpath}/sqltmp/sqltmp.sql | ssh -l $destination_user $destination_host bash -c "'
/usr/bin/mysql -u $prod_db_user --password=\"$prod_db_password\" -h $prod_db_host $prod_db_name'"
Whats happening above? Well we are executing a mysqldump over ssh to our staging server and saving it locally (on the Jenkins server) in a temporary file. This seemed less complicated than sshing to the staging server and executing another mysqldump over ssh to the production server. Kind of makes your head spin!
In any case once the temp SQL file is generated locally over the initial mysql over ssh, we then take that file and pipe the SQL file over ssh to mysql on the destination server. The other reason we are doing mysql over SSH is that typically you wouldnt want to expose mysql publicly on the internet. It is much safer and secure to conduct these transfers over SSH.
Make the necessary adjustments on the production site after the database migration
Once everything has been transferred, we need to make sure that any hard coded URLs from the staging server are adjusted in such a way to accommodate the live site url. Typically in wordpress, things like media asset paths and menu links will automatically adjust based on what the SITE_URL is set in the wp-config.php file. However this isn’t a sure thing especially if you hard code links and image references in your page/post content to the old staging URL.
We utilize the WordPress command line client WP-CLI in our script to accommodate this requirement :
# Get Prod Site Url echo "Fixing URLs on production.." destination_siteurl=$(ssh -l $destination_user $destination_host "cd $destination_dir;wp option get siteurl") staging_siteurl=$(ssh -l $staging_user $staging_host "cd $source_dir;wp option get siteurl")
Whats going on in the above snippet? Well we are getting the absolute destination (production) and staging site urls , as they are configured in wordpress. We are sshing to the respective servers and running a WP-CLI command to get the siteurl, then we are storing the result in a variable to use for the next command :
ssh -l $destination_user $destination_host \
"cd $destination_dir;\
wp search-replace \"$staging_siteurl\" \"$destination_siteurl\" --all-tables --precise;\
wp plugin deactivate shift8-jenkins ;\
wp cache flush"
In the above snippet, we are executing multiple commands over ssh. We are using the WP-CLI utility to execute a search & replace across the site database to replace all instances of the staging URL with the production url. We then want to disable the Jenkins WordPress plugin on the production system and lastly we clear any cache that might be present.
Disable maintenance message
After everything is complete, we want to disable that maintenance message that we switched on in the beginning of our script :
# Disable Maintenance Mode
echo "Disabling maintenance mode.."
ssh -l $destination_user $destination_host \
"cd $destination_dir;\
mv maintenance_on.html maintenance_off.html"
Same as before, but in reverse, we are simply renaming the maintenance mode file which deactivates the nginx trigger and re-activates the site.
In a nutshell, thats what our push process script does! I hope this is helpful. It is definitely a good starting point to integrate your own push process, perhaps to couple with our WordPress jenkins integration script! Some things to expand on in this script would definitely be better error handling and perhaps simple alerting for problems during the script.
You can get the script from GitHub or find the full script below as a reference :
#!/bin/sh
# Jenkins WordPress Push Script
# Star Dot Hosting Inc, 2017
# https://shift8web.ca
#check command input
if [ -z "$1" ];
then
echo "JENKINS WP PUSH"
echo "---------------"
echo ""
echo "Usage : ./jenkins-wordpress.sh sitename.com"
echo ""
exit 0
fi
# Declare variables
currentdate=`date "+%Y-%m-%d"`
scriptpath="/usr/local/bin/jenkins"
# Command arguments
site_name=`echo "$1" | awk -F "." '{printf "%s\n" ,$1}' | sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' | sed 's/-/_/g' | awk -F. '{str="";if ( length($1) > 16 ) str="";print substr($1,0,15)str""$2}'`
# Get configuration variables
source ${scriptpath}/config/${site_name}.conf
# Enable Maintenance Mode
echo "Enabling maintenance mode.."
ssh -l $destination_user $destination_host \
"cd $destination_dir;\
mv maintenance_off.html maintenance_on.html"
# Transfer Files
echo "Transferring files from staging to production.."
ssh -l $staging_user $staging_host \
"cd $source_dir;\
/usr/bin/rsync -rlptDu --exclude='wp-config.php' --exclude='.htaccess' --exclude='shift8-jenkins' --delete ${source_dir}\ ${destination_user}@${destination_host}:${destination_dir}"
# Transfer Database to temp file
echo "Dumping staging database to temp file.."
ssh -l $staging_user $staging_host bash -c "'
/usr/bin/mysqldump -u $staging_db_user --password=\"$staging_db_password\" -h $staging_db_host $staging_db_name
'" > ${scriptpath}/sqltmp/sqltmp.sql
# Transfer Database to production
echo "Transferring staging database to production.."
cat ${scriptpath}/sqltmp/sqltmp.sql | ssh -l $destination_user $destination_host bash -c "'
/usr/bin/mysql -u $prod_db_user --password=\"$prod_db_password\" -h $prod_db_host $prod_db_name'"
# Get Prod Site Url
echo "Fixing URLs on production.."
destination_siteurl=$(ssh -l $destination_user $destination_host "cd $destination_dir;wp option get siteurl --allow-root")
staging_siteurl=$(ssh -l $staging_user $staging_host "cd $source_dir;wp option get siteurl")
ssh -l $destination_user $destination_host \
"cd $destination_dir;\
wp search-replace \"$staging_siteurl\" \"$destination_siteurl\" --all-tables --precise;\
wp plugin deactivate shift8-jenkins;\
wp cache flush"
# Disable Maintenance Mode
echo "Disabling maintenance mode.."
ssh -l $destination_user $destination_host \
"cd $destination_dir;\
mv maintenance_on.html maintenance_off.html"