Manage and deploy Drupal code securely with Git, gitosis and Capistrano
Manage and deploy Drupal code securely with Git, gitosis and Capistrano
UPDATE 2: I have written a follow-up article which simply covers the branching technique described in this post.
UPDATE: I am now using git-http-backend instead of Gitosis. I made the switch because I needed a more central location for managing our projects. I have not written about it yet, but this new approach allows me to authenticate via an LDAP/Active Directory type service.
I have quite a few sites set up with Drupal and it has been working beautifully for me. However, with the fast turn over of Drupal code, I was having trouble keeping all of the sites up to date with the most current code releases. I needed to figure out how to manage my Drupal code in a way that allowed me to upgrade easily without stressing about what I might break. After a lot of research and trial and error, I have finally settled on the following setup.
The main motivation for using Git is the painless branching and merging. I am using Capistrano cause it is easy to set up and work with.
This post is meant to be a tutorial of sorts, but is more likely to be uses as a reference when setting up or changing your sites.
What we will do (or need to do):
- Install Git on your local machine
- Install Git on your repository machine
- Install Git on your staging machine (optional)
- Install Git on your production machine
- Install gitosis on your repository machine
- Setup gitosis
- Setup your sites in Git and push to the repository machine
- Install and setup Capistrano
- Deploy your site to a staging machine (optional)
- Deploy your site to a production machine
Get the machines ready to do some real work…
Install Git on your local machine, repository machine and production machine (and the other machines as needed). This is easy enough, so I will not got into detail. If you have problems, you can checkout these articles because I have gone into installation details there.
Install gitosis on your Repository machine:
$ cd ~/src $ git clone git://eagain.net/gitosis.git $ cd gitosis $ python setup.py install
If you get errors on the above line of code, then you will need to install python-setuptools.
Now set up a user on your Repository machine that will be the one who owns the repositories. I use ‘git’.
$ sudo adduser -system -shell /bin/sh -gecos 'git version control' -group -disabled-password -home /home/git git
Now you need to have an SSH Key set up on your local machine. If you don’t have one, do this (on your local machine)…
$ ssh-keygen -t rsa
Assuming that you created the file
id_rsa.pub, now you need to copy it to
your Repository machine. I put mine in
/tmp/id_rsa.pub so it is easy to
Now initialize gitosis with that key (on the Repository machine).
$ sudo -H -u git gitosis-init < /tmp/id_rsa.pub
Make sure that
post-update is set to executable.
$ sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update
To set up the gitosis admin on your local machine.
(Everything from here on is on your local machine…)
cd into the directory you want your code to live first.
$ git clone git@YOUR_REPOSITORY_MACHINE:gitosis-admin.git $ cd gitosis-admin
Setup the admin for the repositories.
Open the file
gitosis.conf and make sure it has the following for the admin
[gitosis] [group gitosis-admin] writable = gitosis-admin members = youruser@machine
(youruser@machine is going to be taken from your
id_rsa.pub file. it will be
the last piece of that file and will look something like: hzvu4nTtw3Q==
UPDATE: I have had some problems with this method of specifying a user. What I do now, which seems to work, is make sure that my .pub files have unique names when I create them (eg: __ which could be something like, forwardthinkingdesign_prod_root) and I use the filename in the ‘members’ section.
Setup and manage your repositories…
gitosis.conf add your project (your_project in this example).
[group yourteam] members = youruser@machine writable = your_project
Push this updated configuration to the server.
(you will probably have to add the appropriate files to be committed with
add. when you have more than one project, you will have more files here that
will be untracked.)
$ git commit -a -m "Allow youruser@machine write access to your_project" $ git push
Now you are going to actually create
your_project and push it to the gitosis
$ mkdir your_project $ cd your_project $ git init $ git remote add origin git@REPOSITORY_MACHINE:your_project.git
At this point you need to setup the how you are going to manage your Drupal
code. You can put it all in one branch with
git push origin
master:refs/heads/master, but I would NOT recommend this.
The Git setup I recommend for managing Drupal projects.
Create a branch
drupal just for the Drupal code. You never make changes to
this branch other than upgrading to newer versions of Drupal. Git created a
master branch when you did
git init and since Git does not like you trying
to branch from it without doing any ‘real work’, we are going to add to the
master branch and then rename it to
drupal before we add it to the
$ tar xvzf drupal-6.x.tar.gz $ rm -rf drupal-6.x.tar.gz $ mv drupal-6.x drupal
I also add a
.gitignore file to this base
drupal branch so I do not have
to worry about stuff I do not want getting into my repository.
.gitignore file in the root of your repository and add the
following lines to it.
.DS_Store drupal/sites/default/files/ Capfile config/deploy.rb
Explanation of ignores:
_.DSStore -> This is because I am on a Mac and it creates these files all over the place. They should not be in my repository.
drupal/sites/default/files/ -> This is because I run a local server on my machine that needs the ‘files’ directory, but I do not want to track the ‘files’ directory in my repo. (this may also be drupal/files/ depending on what version of drupal you are on.)
Capfile -> We have not gotten this far, but this is for Capistrano.
config/deploy.rb -> Again, this is for Capistrano.
Now add and commit these changes.
$ git add . $ git commit -a -m "Initial Drupal commit"
master branch to our
$ git branch -m master drupal
We now have the
drupal branch up to date on our local machine. Lets push it
to the repository server.
$ git push origin drupal:refs/heads/drupal
drupal branch we want to create a
modules branch that will be
used to manage all of the unmodified contributed modules. This should only be
the module code that you get from drupal.org. You do not modify any of the
code in this branch, just upgrade the modules when needed.
You need to make sure you are on the
drupal branch and then you will create
modules branch. Once there you will add all your modules to the
$ git checkout drupal $ git checkout -b modules $ cd drupal/sites/all $ mkdir modules $ cd modules $ tar xzvf yourmodule.tar.gz $ rm -rf yourmodule.tar.gz $ ... (repeat for all your modules) ... $ git add . $ git commit -a -m "Initial add of all of my sites modules" # I also track my themes that are not modified in this branch
When you are done adding all of your modules then you will want to push the
modules branch to the repository machine.
$ git push origin modules:refs/heads/modules
Now you should have all of the base modules (and themes) tracked in your
repository. At this point you want a branch where you can do all your
modifications that are custom to your site. I track all my custom site
specific modules, custom site specific themes and all of my module and core
hacks in this branch. I call this branch
production since it is what I push
out to my staging and production machines.
Lets create the
production branch and add our site specific stuff (including
$ git checkout modules $ git checkout -b production # make all of your changes that are custom to the site. $ git add . $ git commit -a -m "Initial add of all of my site specific hacks/modules/themes/etc..."
Now lets push this branch to the repository machine.
$ git push origin production:refs/heads/production
Updating to a new version of Drupal (or updating modules)
This is where the real power of Git comes in and this is why I use this method.
When upgrading the Drupal core, we only want to change the
because that is where the Drupal core is stored.
drupal branch and once there we are going to replace the
current version of Drupal with a new version.
$ git checkout drupal $ rm -rf drupal $ tar xvzf drupal-6.xx.tar.gz $ rm -rf drupal-6.xx.tar.gz $ mv drupal-6.xx drupal
Now lets update the local repository and then push it to the repository machine.
$ git add . $ git commit -a -m "Drupal 6.xx update" $ git push origin drupal:refs/heads/drupal
We have to propagate this change up through our other branches, so we will
pull these changes into the
modules branch and then commit and push it to
the repository machine.
$ git checkout modules $ git pull . drupal $ git push origin modules:refs/heads/modules
And likewise for the
$ git checkout production $ git pull . modules $ git push origin production:refs/heads/production
Now all of your code is up to date on your local machine as well as your
repository machine. You can update the
modules branch just the same my
downloading newer versions of the modules code and just update the
and then the
Deploying your code to the staging and production machines
At this point some people may choose to just copy and paste the code from their repository to their staging/production machines. As a simple solution, that is not too bad, but I want a more elegant method for deploying. For this I use Capistrano. Capistrano is built on Ruby and is very popular with Ruby on Rails developers. In order for it to work nicely with PHP and Drupal, we need to create our own
config/deploy.rb file. Luckily for you, I have already
done this and you can find it attached at the end of this article. Lets get
Setup your remote machines with permission to access your repository
In order for you staging and production machines to be allowed to access your repository machine to get the latest code, you need to setup an rsa key for each of them. Do the following on all the machines that need to access the repository (staging and production).
$ ssh YOUR_USER@REMOTE_MACHINE $ ssh-keygen -t rsa # Be sure to create it with a unique name (example: 'staging')
staging.pub file from REMOTE_MACHINE to your local machine and
place it in the
gitosis-admin/keydir folder. You need to edit the
admin/gitosis.conf file and add the user from your REMOTE_MACHINE in the
appropriate section as we did before.
Example (added the bold):
[group yourteam] members = youruser@machine **remoteuser@remotemachine** writable = your_project
Now add, commit and push the changes to the repository machine (needs to be
done from inside the
$ git add gitosis.conf keydir/staging.pub $ git commit -a -m "Gave the staging machine the ability to access the repository" $ git push
(Repeat for all the machines that need access to the repository)…
Giving additional users access to your repository
If you need to add someone to the project, set them up with access the same way you just did above for the ‘staging’ machine. Once they are setup with permissions, they will need to clone the project to have access.
$ git clone git@REPOSITORY_MACHINE:your_project.git
Notes - START
If you are on Mac OS X Snow Leopard, it may ask you for a password. This is because Snow Leopard ships with key forwarding disabled by default and you will have to modify the file
/etc/ssh_config to get it working.
Change the lines:
# Host * # ForwardAgent no
Host * ForwardAgent yes
If it still asks you for a password on Snow Leopard, you may need to add your passphrases to the Apple keychain. Type the following in a terminal.
$ ssh-add -K ~/.ssh/name_of_your_key
If you still have problems, you may want to add
loglevel = DEBUG under the
[gitosis] section of your
gitosis.conf file to get more information about
what is happening.
This often happens and is a PITA to figure out. One thing that I have found is that once you add a user to the remote repository, they have problems connection to the remote machine to clone the repository. This may resolve the issue (this has always been on a Mac).
Add the following to the file:
Host REPOSITORY_MACHINE User git Hostname REPOSITORY_MACHINE PreferredAuthentications publickey IdentityFile /path/to/.ssh/filename
(filename is the same as the filename.pub file, but without the .pub at the end)
Notes - END
Once it clones the project, you will probably get the following error:
Warning: Remote HEAD refers to nonexistent ref, unable to checkout.
This is basically saying “I don’t know what branch to checkout”.
Check and see what branches you have available:
$ cd your_project $ git branch -a
This should show you:
origin/drupal origin/modules origin/production
You will not want to check those branches out directly because they are remote branches and you will detach the head if you do. Basically you want to create local branches that you can change that will be linked to the remote branches.
$ git checkout -b drupal origin/drupal $ git checkout -b modules origin/modules $ git checkout -b production origin/production
Now they will have a working local copy of the repository and they will be able to make commits to the remote repository.
Setup Capistrano and server configuration
You need to install Capistrano on your local machine. They have great documentation at the Capistrano website which I reference every time I do this, so I will leave that as an exercise for you to do.
At this point I am assuming that you have Capistrano installed on your local machine. We now need to setup our project to deploy with Capistrano.
In the root of your repository directory (eg:
your_project) on your local
machine, do this.
$ mkdir config $ capify .
Download the file (
deploy.rb) that is attached and replace the
file in the
config directory. You will have to go through the file and
change all the sections that are IN_CAPS to the correct information. I will
try to make this painless, but it may take some playing to get it setup for
your environment. A lot of this depends on how you want to setup your server,
so you may have to change the configuration as needed…
Now that your configuration is correct we are going to do the setup.
Note: You need to setup your staging and production machines the same as you setup in the deploy.rb file. So you need to create the directory that you want to deploy to (in my example it is:
/var/www/, and make sure it is owned by the user that you specified in your deploy.rb file).
Now we need to create the skeleton of your deploy file structure.
$ cap deploy:setup
Make sure that you have everything in place to do the deploy.
$ cap deploy:check
(If you already have ‘files’ for your project, you may want to copy them into
/var/www/your_project/shared/files/ directory at this point so the
actual deploy command can change the permissions correctly.)
Lets do the deployment…
$ cap deploy
You have just deployed your code from your repository machine to you staging
or production machine. You will have to make sure that your apache
configuration is setup correctly to serve the site that is currently deployed.
To do that I have attached a sample vhosts file (
sample_site.conf) that may
get you moving in the right direction.
Congratulations!!! Now you can hack the core to your hearts content and you will still have a clean upgrade path (just do it in your
production branch). ;)
- Hosting Git repositories, The Easy (and Secure) Way
- Drupal Development and Deployment using Git
- Installing gitosis
- A Tempest of Thoughts - Capistrano
- Enable SSH Agent (Key) Forwarding on Snow Leopard