Chapter 6. Migrating to Git

In this chapter, we will try to migrate a Subversion repository into a Git one, preserving the changes history. Git and Subversion can coexist as Git has some dedicated commands to exchange data with Subversion, and you can even continue to use both.

The purpose of this chapter is to help developers who actually use Subversion to start using Git instantly, even if the rest of the team continues to use Subversion. In addition, the chapter covers definitive migration for people who decide to abandon Subversion in favor of Git.

Before starting

In the first part of this chapter, we will take a look at some good practices to keep safety and work on actual SVN repository with no hassles. Bear in mind that the purpose of this chapter is only to give readers some hints; dealing with big and complex repositories deserves a more prudent and articulated approach.

Prerequisites

To be able to do these experiments, you need a Subversion tool; on Windows, the most used tool is the well-known TortoiseSVN (available at http://tortoisesvn.net), which provides both command-line tools: GUI and shell integration.

I recommend to do a full installation of TortoiseSVN, including command-line tools as we'll need some of them to make experiments.

Working on a Subversion repository using Git

In the first part of this chapter, we will see the most cautious approach while starting to move away from Subversion, which is to keep the original repository using Git to fetch and push changes. For the purpose of learning, we will create a local Subversion repository, using both Subversion and Git to access its contents.

Creating a local Subversion repository

Without the hassle of remote servers, let's create a local Subversion repository as a container for our experiments:

$ cd \Repos $ svnadmin create MySvnRepo

Now there's nothing more and nothing less to be done here, and the repository is now ready to be filled with folders and files.

Checking out the Subversion repository with svn client

At this point, we have a working Subversion repository; we can now check it out in a folder of our choice, which will become our working copy; in my case, I will use the C:\Sources folder:

$ cd \Sources\svn $ svn checkout file:///Repos/MySvnRepo

You now have a MySvnRepo folder under the Sources folder, ready to be filled with your project files; but first let me remind you of a couple of things.

As you may know, a Subversion repository generally has the following subfolder structure:

• /trunk: This is the main folder, where generally you have the code under development
• /tags: This is the root folder of the snapshots you usually freeze and leave untouched, for example /tags/v1.0
• /branches: This is the root folder of all the repository branches you will create for feature development, for example /branches/NewDesign

Subversion does not provide a command to initialize a repository with this layout (commonly known as standard layout), so we have to build it up by hand.

At this point, we can import a skeleton folder that already contains the three subfolders (/trunk, /branches, and /tags) with a command like this:

$ cd \Sources\svn\MySvnRepo
$ svn import /path/to/some/skeleton/dir

Otherwise, we can create folders by hand using the svn mkdir command:

$ cd \Sources\svn\MySvnRepo
$ svn mkdir trunk
$ svn mkdir tags
$ svn mkdir branches

Commit the folders we just created and the repository is ready:

svn commit -m "Initial layout"

Now add and commit the first file, as shown in the following code:

$ cd trunk
$ echo "This is a Subversion repo" > readme.txt
$ svn add readme.txt
$ svn commit -m "Readme file"

Feel free to add more files or import an existing project if you want to replicate a more real situation; for import files in a Subversion repository, you can use the svn import command, as you saw before:

$ svn import \MyProject\Folder

Later, we will add a tag and a branch to verify how Git interacts with them.

Cloning a Subversion repository from Git

Git provides a set of tools to cooperate with Subversion; the base command is actually git svn; with git svn you can clone Subversion repositories, retrieve and upload changes, and much more.

So, wear the Git hat and clone the Subversion repository using the git svn clone command:

$ cd \Sources\git
$ git svn clone file:///Repos/MySvnRepo

git svn clone usually runs smoothly on Linux boxes, while in Windows you get a weird error message, as shown here:

Couldn't open a repository: Unable to open an ra_local session to URL

This happens because in Windows there are some problems in the git svn command backport while using the file:// protocol.

Setting up a local Subversion server

To bypass this problem, we can use the svn:// protocol instead of file://.

To do this, we will use the svnserve command to start a Subversion server exposing our repositories directory root. But first we have to edit some files to set up the server.

The main file is the svnserve.conf file; look at your MySvnRepo repository folder, C:\Repos\MySvnRepo in this case, jump into the conf subfolder, and edit the file uncommenting these lines (remove the starting #):

anon-access = read
auth-access = write
password-db = C:\Repos\MySvnRepo\passwd
authz-db = C:\Repos\MySvnRepo\authz
realm = MySvnRepo

Normally, full paths are unnecessary for the passwd and authz files, but in Windows I find them mandatory, otherwise the Subversion server does not load them.

Now edit the passwd file, which contains the user's credentials, and add a new user:

[users]
adminUser = adminPassword

Then edit the authz file and set up groups and rights for the user:

[groups]
Admins = admin

Continuing with the authz file, set rights for repositories; the Admin will have read/write access to all repositories (the section [/] means "all repository"):

[/]
@Admins = rw
* =

At this point, we can start the server with this command:

$ svnserve -d --config-file C:\Repos\MySvnRepo\conf\svnserve.conf --root C:\Repos

-d starts the server as a daemon; --config-file specifies the config file to load and --root tells the server where the main root folder of all your repositories is.

Now we have a Subversion server responding at svn://localhost:3690. We can finally clone the repository using Git:

$ cd \Sources\git
$ git svn clone svn://localhost/MySvnRepo/trunk

This time things will go smoothly; we are now talking with a Subversion server using Git.

Adding a tag and a branch

Just to have a more realistic situation, I will add a tag and a branch; in this manner, we will see how to deal with them in Git.

So, add a new file:

$ echo "This is the first file" > svnFile01.txt
$ svn add svnFile01.txt
$ svn commit -m "Add first file"

Then tag this snapshot of the repository as v1.0 as you know that in Subversion, a tag or a branch is a copy of a snapshot:

$ echo "This is the first file" > svnFile01.txt
$ svn add svnFile01.txt
$ svn copy svn://localhost/MySvnRepo 
svn://localhost/MySvnRepo/tags/v1.0 -m "Release 1.0"

Committing a file to Subversion using Git as a client

Now that we have a running clone of the original Subversion repository, we can use Git as it was a Subversion client. So add a new file and commit it using Git:

$ echo "This file comes from Git" >> gitFile01.txt
$ git add gitFile01.txt
$ git commit –m "Add a file using Git"

Now we have to push this file to Subversion:

$ git svn dcommit

Well done! We can even use Git to fetch changes with the git svn fetch command, update the local code using the git svn rebase command, and so on; for other commands and options, I recommend that you read the main page of git svn --help.

Using Git as a Subversion client is not the best we can obtain, but at least it is a way to start using Git even if you cannot abandon Subversion instantly.

Using Git with a Subversion repository

Using Git as a client of Subversion can raise some confusion due to the flexibility of Git as compared to the more rigid way Subversion organizes files. To be sure to maintain a Subversion-friendly way of work, I recommend that you follow some simple rules.

First of all, be sure your Git master branch is related to the trunk branch in Subversion; as we already said, Subversion users usually organize a repository in this way:

• a /trunk folder, which is the main folder
• a /branches root folder, where you put all the branches, each one located in a separate subfolder (for example, /branches/feat-branch)
• a /tags root folder, where you collect all the tags you made (for example, /tags/v1.0.0)

To adhere to this layout, you can use the --stdlayout option when you're cloning a Subversion repository:

$ git svn clone <url> --stdlayout

In this manner, Git will hook the /trunk Subversion branch to the Git master branch, replicating all the /branches and /tags branches in your local Git repository and allowing you to work with them in a 1:1 synchronized context.

Migrating a Subversion repository

When possible, it is recommended to completely migrate a Subversion repository to Git; this is quite simple to do and mostly depends on the size of the Subversion repository and the organization.

If the repository follows the standard layout as described before, a migration is only a matter of minutes.

Retrieving the list of Subversion users

If your Subversion repository has been used from different people, you are probably interested in preserving the commit author's name, which is true even in the new Git repository.

If you have the awk command available (maybe using Cygwin in Windows), there is a script here that fetches all the users from Subversion logs and appends them to a text file we can use in Git while cloning to perfectly match the Subversion users, even in Git-converted commits:

$ svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt

Now we will use the authors.txt file in the next cloning step.

Cloning the Subversion repository

To begin the migration, we have to locally clone the Subversion repository as we did before; I recommend once more adding the --stdlayout option to preserve the branches and tags and then to add the -A option to let Git convert commit authors while cloning:

$ git svn clone <repo-url> --stdlayout –-prefix svn/ -A authors.txt

In case the Subversion repository has trunks, branches, and tags located in other paths (with no standard layout), Git provides you with a way to specify them with the --trunk, --branches, and --tags options:

$ git svn clone <repo-url> --trunk=<trunk-folder> --branches=<branches-subfolder> --tags=<tags-subfolder>

When you fire the clone command, remember that this operation can be time- consuming; in a repository with 1000 commits, it is not unusual to wait 15 to 30 minutes for this.

Preserving the ignored file list

To preserve the previously ignored files in Subversion, we can append the svn:ignore settings to the .gitignore file:

$ git svn show-ignore >> .gitignore
$ git add .gitignore
$ git commit -m "Convert svn:ignore properties to .gitignore"

Pushing to a local bare Git repository

Now that we have a local copy of our repository, we can move it to a brand new Git repository. Here you can already use a remote repository on your server of choice, which can even be GitHub or Bitbucket, but I recommend that you use a local bare repository; we may as well do some other little adjustments (like renaming tags and branches) before pushing files to a blessed repository. So, first initialize a bare repository in a folder of your choice:

$ mkdir \Repos\MyGitRepo.git
$ cd \Repos\MyGitRepo.git
$ git init --bare

Now make the default branch to match the Subversion trunk branch name:

$ git symbolic-ref HEAD refs/heads/trunk

Then add a bare remote pointing to the bare repository just created:

$ cd \Sources\MySvnRepo
$ git remote add bare file:///C/Repos/MyGitRepo.git

Finally push the local cloned repository to the new bare one:

$ git push --all bare

We now have a brand new bare repository that is a perfect copy of the original Subversion repository. We can now adjust branches and tags to better fit the usual Git layout.

Arranging branches and tags

Now we can rename branches and tags to obtain a more Git-friendly scenario.

Renaming the trunk branch to master

The Subversion main development branch is /trunk, but in Git, as you know, we prefer to call the main branch master; here's a way to rename it:

$ git branch -m trunk master

Converting Subversion tags to Git tags

Subversion treats tags as branches; they are all copies of a certain trunk snapshot. In Git, on the contrary, branches and tags have a different significance.

To convert Subversion tags and branches into Git tags, the following simple script does the work:

$ git for-each-ref --format='%(refname)' refs/heads/tags |
cut -d / -f 4 |
while read ref
do
 git tag "$ref" "refs/heads/tags/$ref";
 git branch -D "tags/$ref";
done

Pushing the local repository to a remote

You now have a local bare Git repository ready to be pushed to a remote server; the result of the conversion is a full Git repository, where branches, tags, and commit history have been preserved. The only thing you have to do by hand is to eventually accommodate Git users.

Comparing Git and Subversion commands

Here you can find a short and partial recap table, where I try to pair the most common Subversion and Git commands to help Subversion users to quickly shift their minds from Subversion to Git.

Summary

This chapter barely scratches the surface, but I think it'll be useful to get a sense of the topic. If you have wide Subversion repositories, you will probably need better training before starting to convert them to Git, but for small to medium ones, now you know the fundamentals to begin with.

The only suggestion I want to share with you is to not be in a hurry; start by letting Git cooperate with your Subversion server, reorganize your repository when it's messy, do a lot of backups, and finally try to convert it; you will convert it more than once, as I did, but in the end you will get more satisfaction from doing that.

In the next chapter, I will share with you some useful resources I found during my career as a Git user.