Manage and deploy Drupal code securely with Git, gitosis and Capistrano
Manage and deploy Drupal code securely with Git, gitosis and Capistrano
UPDATE 2: I have written a follow-up article which simply covers the branching technique described in this post.
UPDATE: I am now using git-http-backend instead of Gitosis. I made the switch because I needed a more central location for managing our projects. I have not written about it yet, but this new approach allows me to authenticate via an LDAP/Active Directory type service.
I have quite a few sites set up with Drupal and it has been working beautifully for me. However, with the fast turn over of Drupal code, I was having trouble keeping all of the sites up to date with the most current code releases. I needed to figure out how to manage my Drupal code in a way that allowed me to upgrade easily without stressing about what I might break. After a lot of research and trial and error, I have finally settled on the following setup.
The main motivation for using Git is the painless branching and merging. I am using Capistrano cause it is easy to set up and work with.
This post is meant to be a tutorial of sorts, but is more likely to be uses as a reference when setting up or changing your sites.
What we will do (or need to do):
- Install Git on your local machine
- Install Git on your repository machine
- Install Git on your staging machine (optional)
- Install Git on your production machine
- Install gitosis on your repository machine
- Setup gitosis
- Setup your sites in Git and push to the repository machine
- Install and setup Capistrano
- Deploy your site to a staging machine (optional)
- Deploy your site to a production machine
Get the machines ready to do some real work…
Install Git on your local machine, repository machine
and production machine (and the other machines as needed). This is easy
enough, so I will not got into detail. If you have problems, you can checkout
these articles
because I have gone into installation details there.
Install gitosis on your Repository machine:
$ cd ~/src
$ git clone git://eagain.net/gitosis.git
$ cd gitosis
$ python setup.py install
If you get errors on the above line of code, then you will need to install python-setuptools.
Now set up a user on your Repository machine that will be the one who owns the repositories. I use ‘git’.
$ sudo adduser -system -shell /bin/sh -gecos 'git version control' -group -disabled-password -home /home/git git
Now you need to have an SSH Key set up on your local machine. If you don’t have one, do this (on your local machine)…
$ ssh-keygen -t rsa
Assuming that you created the file id_rsa.pub
, now you need to copy it to
your Repository machine. I put mine in /tmp/id_rsa.pub
so it is easy to
access.
Now initialize gitosis with that key (on the Repository machine).
$ sudo -H -u git gitosis-init < /tmp/id_rsa.pub
Make sure that post-update
is set to executable.
$ sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update
To set up the gitosis admin on your local machine.
(Everything from here on is on your local machine…)
Note: cd
into the directory you want your code to live first.
$ git clone git@YOUR_REPOSITORY_MACHINE:gitosis-admin.git
$ cd gitosis-admin
Setup the admin for the repositories.
Open the file gitosis.conf
and make sure it has the following for the admin
setup.
[gitosis]
[group gitosis-admin]
writable = gitosis-admin
members = youruser@machine
(youruser@machine is going to be taken from your id_rsa.pub
file. it will be
the last piece of that file and will look something like: hzvu4nTtw3Q==
youruser@machine)
UPDATE: I have had some problems with this method of specifying a user. What I do now, which seems to work, is make sure that my .pub files have unique names when I create them (eg: __ which could be something like, forwardthinkingdesign_prod_root) and I use the filename in the ‘members’ section.
Setup and manage your repositories…
Also in gitosis.conf
add your project (your_project in this example).
[group yourteam]
members = youruser@machine
writable = your_project
Push this updated configuration to the server.
(you will probably have to add the appropriate files to be committed with git
add
. when you have more than one project, you will have more files here that
will be untracked.)
$ git commit -a -m "Allow youruser@machine write access to your_project"
$ git push
Now you are going to actually create your_project
and push it to the gitosis
repository.
$ mkdir your_project
$ cd your_project
$ git init
$ git remote add origin git@REPOSITORY_MACHINE:your_project.git
At this point you need to setup the how you are going to manage your Drupal
code. You can put it all in one branch with git push origin
master:refs/heads/master
, but I would NOT recommend this.
The Git setup I recommend for managing Drupal projects.
Create a branch drupal
just for the Drupal code. You never make changes to
this branch other than upgrading to newer versions of Drupal. Git created a
master
branch when you did git init
and since Git does not like you trying
to branch from it without doing any ‘real work’, we are going to add to the
master
branch and then rename it to drupal
before we add it to the
repository machine.
$ tar xvzf drupal-6.x.tar.gz
$ rm -rf drupal-6.x.tar.gz
$ mv drupal-6.x drupal
I also add a .gitignore
file to this base drupal
branch so I do not have
to worry about stuff I do not want getting into my repository.
Create a .gitignore
file in the root of your repository and add the
following lines to it.
.DS_Store
drupal/sites/default/files/
Capfile
config/deploy.rb
Explanation of ignores:
_.DSStore -> This is because I am on a Mac and it creates these files all
over the place. They should not be in my repository.
drupal/sites/default/files/ -> This is because I run a local server on my
machine that needs the ‘files’ directory, but I do not want to track the
‘files’ directory in my repo. (this may also be drupal/files/ depending on
what version of drupal you are on.)
Capfile -> We have not gotten this far, but this is for Capistrano.
config/deploy.rb -> Again, this is for Capistrano.
Now add and commit these changes.
$ git add .
$ git commit -a -m "Initial Drupal commit"
Rename the master
branch to our drupal
branch.
$ git branch -m master drupal
We now have the drupal
branch up to date on our local machine. Lets push it
to the repository server.
$ git push origin drupal:refs/heads/drupal
From the drupal
branch we want to create a modules
branch that will be
used to manage all of the unmodified contributed modules. This should only be
the module code that you get from drupal.org. You do not modify any of the
code in this branch, just upgrade the modules when needed.
You need to make sure you are on the drupal
branch and then you will create
a modules
branch. Once there you will add all your modules to the modules
branch.
$ git checkout drupal
$ git checkout -b modules
$ cd drupal/sites/all
$ mkdir modules
$ cd modules
$ tar xzvf yourmodule.tar.gz
$ rm -rf yourmodule.tar.gz
$ ... (repeat for all your modules) ...
$ git add .
$ git commit -a -m "Initial add of all of my sites modules"
# I also track my themes that are not modified in this branch
When you are done adding all of your modules then you will want to push the
modules
branch to the repository machine.
$ git push origin modules:refs/heads/modules
Now you should have all of the base modules (and themes) tracked in your
repository. At this point you want a branch where you can do all your
modifications that are custom to your site. I track all my custom site
specific modules, custom site specific themes and all of my module and core
hacks in this branch. I call this branch production
since it is what I push
out to my staging and production machines.
Lets create the production
branch and add our site specific stuff (including
drupal/sites/default/settings.php).
$ git checkout modules
$ git checkout -b production
# make all of your changes that are custom to the site.
$ git add .
$ git commit -a -m "Initial add of all of my site specific hacks/modules/themes/etc..."
Now lets push this branch to the repository machine.
$ git push origin production:refs/heads/production
Updating to a new version of Drupal (or updating modules)
This is where the real power of Git comes in and this is why I use this
method.
When upgrading the Drupal core, we only want to change the drupal
branch
because that is where the Drupal core is stored.
Checkout the drupal
branch and once there we are going to replace the
current version of Drupal with a new version.
$ git checkout drupal
$ rm -rf drupal
$ tar xvzf drupal-6.xx.tar.gz
$ rm -rf drupal-6.xx.tar.gz
$ mv drupal-6.xx drupal
Now lets update the local repository and then push it to the repository machine.
$ git add .
$ git commit -a -m "Drupal 6.xx update"
$ git push origin drupal:refs/heads/drupal
We have to propagate this change up through our other branches, so we will
pull these changes into the modules
branch and then commit and push it to
the repository machine.
$ git checkout modules
$ git pull . drupal
$ git push origin modules:refs/heads/modules
And likewise for the production
branch.
$ git checkout production
$ git pull . modules
$ git push origin production:refs/heads/production
Now all of your code is up to date on your local machine as well as your
repository machine. You can update the modules
branch just the same my
downloading newer versions of the modules code and just update the modules
and then the production
branches.
Deploying your code to the staging and production machines
At this point some people may choose to just copy and paste the code from
their repository to their staging/production machines. As a simple solution,
that is not too bad, but I want a more elegant method for deploying. For this
I use Capistrano. Capistrano is built on Ruby and is very popular with Ruby on
Rails developers. In order for it to work nicely with PHP and Drupal, we need
to create our own config/deploy.rb
file. Luckily for you, I have already
done this and you can find it attached at the end of this article. Lets get
started…
Setup your remote machines with permission to access your repository
In order for you staging and production machines to be allowed to access your
repository machine to get the latest code, you need to setup an rsa key for
each of them. Do the following on all the machines that need to access the
repository (staging and production).
$ ssh YOUR_USER@REMOTE_MACHINE
$ ssh-keygen -t rsa
# Be sure to create it with a unique name (example: 'staging')
Copy the staging.pub
file from REMOTE_MACHINE to your local machine and
place it in the gitosis-admin/keydir
folder. You need to edit the gitosis-
admin/gitosis.conf
file and add the user from your REMOTE_MACHINE in the
appropriate section as we did before.
Example (added the bold):
[group yourteam]
members = youruser@machine **remoteuser@remotemachine**
writable = your_project
Now add, commit and push the changes to the repository machine (needs to be
done from inside the gitosis-admin
directory).
$ git add gitosis.conf keydir/staging.pub
$ git commit -a -m "Gave the staging machine the ability to access the repository"
$ git push
(Repeat for all the machines that need access to the repository)…
Giving additional users access to your repository
If you need to add someone to the project, set them up with access the same
way you just did above for the ‘staging’ machine. Once they are setup with
permissions, they will need to clone the project to have access.
$ git clone git@REPOSITORY_MACHINE:your_project.git
Notes - START
If you are on Mac OS X Snow Leopard, it may ask you for a password. This is because Snow Leopard ships with key forwarding disabled by default and you will have to modify the file /etc/ssh_config
to get it working.
Change the lines:
# Host *
# ForwardAgent no
To:
Host *
ForwardAgent yes
If it still asks you for a password on Snow Leopard, you may need to add your passphrases to the Apple keychain. Type the following in a terminal.
$ ssh-add -K ~/.ssh/name_of_your_key
If you still have problems, you may want to add loglevel = DEBUG
under the
[gitosis]
section of your gitosis.conf
file to get more information about
what is happening.
Permission Denied
This often happens and is a PITA to figure out. One thing that I have found is
that once you add a user to the remote repository, they have problems
connection to the remote machine to clone the repository. This may resolve the
issue (this has always been on a Mac).
Add the following to the file: ~/.ssh/config
Host REPOSITORY_MACHINE
User git
Hostname REPOSITORY_MACHINE
PreferredAuthentications publickey
IdentityFile /path/to/.ssh/filename
(filename is the same as the filename.pub file, but without the .pub at the end)
Notes - END
Once it clones the project, you will probably get the following error:
Warning: Remote HEAD refers to nonexistent ref, unable to checkout.
This is basically saying “I don’t know what branch to checkout”.
Check and see what branches you have available:
$ cd your_project
$ git branch -a
This should show you:
origin/drupal
origin/modules
origin/production
You will not want to check those branches out directly because they are remote branches and you will detach the head if you do. Basically you want to create local branches that you can change that will be linked to the remote branches.
$ git checkout -b drupal origin/drupal
$ git checkout -b modules origin/modules
$ git checkout -b production origin/production
Now they will have a working local copy of the repository and they will be able to make commits to the remote repository.
Setup Capistrano and server configuration
You need to install Capistrano on
your local machine. They have great documentation at the Capistrano website
which I reference every time I do this, so I will leave that as an exercise
for you to do.
At this point I am assuming that you have Capistrano installed on your local machine. We now need to setup our project to deploy with Capistrano.
In the root of your repository directory (eg: your_project
) on your local
machine, do this.
$ mkdir config
$ capify .
Download the file (deploy.rb
) that is attached and replace the deploy.rb
file in the config
directory. You will have to go through the file and
change all the sections that are IN_CAPS to the correct information. I will
try to make this painless, but it may take some playing to get it setup for
your environment. A lot of this depends on how you want to setup your server,
so you may have to change the configuration as needed…
Now that your configuration is correct we are going to do the setup.
Note: You need to setup your staging and production machines the same as you setup in the deploy.rb file. So you need to create the directory that you want to deploy to (in my example it is: /var/www/
, and make sure it is owned by the user that you specified in your deploy.rb file).
Now we need to create the skeleton of your deploy file structure.
$ cap deploy:setup
Make sure that you have everything in place to do the deploy.
$ cap deploy:check
(If you already have ‘files’ for your project, you may want to copy them into
the /var/www/your_project/shared/files/
directory at this point so the
actual deploy command can change the permissions correctly.)
Lets do the deployment…
$ cap deploy
You have just deployed your code from your repository machine to you staging
or production machine. You will have to make sure that your apache
configuration is setup correctly to serve the site that is currently deployed.
To do that I have attached a sample vhosts file (sample_site.conf
) that may
get you moving in the right direction.
Congratulations!!! Now you can hack the core to your hearts content and you will still have a clean upgrade path (just do it in your production
branch). ;)
References:
- Hosting Git repositories, The Easy (and Secure) Way
- Drupal Development and Deployment using Git
- Installing gitosis
- A Tempest of Thoughts - Capistrano
- Enable SSH Agent (Key) Forwarding on Snow Leopard
Attachments:
deploy.rb 3.16 KB
sample_site.conf 864 bytes