Azure DevOps, Scrum, & .NET Software Leadership and Consulting Services

Free course! Predicting the Future, Estimating, and Running Your Projects with Flow Metrics

One TFS Build, Multiple Git Repositories with Submodules


In the discussion thread for my DevOps Skills class at Pluralsight, I got a question about how to do TFS Builds and TFS Release Management with code that spans multiple Git repositories.  For good reasons, his team has separated their code into multiple repos.  But when they go to create a build in TFS, the build templates only know about a single Git repository.  So how do you pull that “best practice” Git repo structure into one unified automated build and create something that can be released?

Why Multiple Git Repositories?

If you’re a TFS with TFVC developer, you never really worried too much about how big your version control repository got.  TFVC (aka. Team Foundation Version Control) is fine with giant repositories.  But when you make the move to Git, you’ll probably start getting a little more thoughtful about how big your Git repository is getting.  Since cloning a git repo puts the entire version control history on everyone’s disk — well, it can get big and you definitely care.  TFVC (or any other centralize version control) only give you a single version of the code on your workstation at once.  Git gives you everything.

There’s the obvious stuff you should avoid doing if you care about about keeping your Git repo size to something manageable.  Don’t put dependencies in the repo (use NuGet or something like NuGet instead).  Don’t put your whole photo library or a bunch of zip files or Word docs or Excel spreadsheets into the repo.  Basically, keep non-mergable files outside of the repo.

But what about your big sprawling code bases?  Surely you can put all that into a single Git repository, right?  Maybe.  But also maybe not.  It’s pretty easy for your applications (especially enterprise apps and line-of-business apps) to get HUGE.  Chances are very high that when you’re working on that app that you don’t need the whole thing all the time.

If your codebase is getting big, you might want to consider multiple Git repositories.  But how do you bring this all together?

Git Submodules

The answer to this problem is Git Submodules.  A submodule is a reference from one Git repository to one or more other repositories.  This helps you to keep stuff organized without having to have one huge, giant git repo that rules the universe.

Assuming that you’ve got two Git repositories: Repo1 and Repo2.  Let’s say that they’re hosted on Visual Studio Team Services (VSTS).  They both have URLs:

  • https://{projectname}.visualstudio.com/DefaultCollection/{team-project-name}/_git/repo1
  • https://{projectname}.visualstudio.com/DefaultCollection/{team-project-name}/_git/repo2

Let’s say that the code in Repo1 needs the code in Repo2 in order to be released.  So you clone repo1 by going to the command line and typing:

git clone https://{projectname}.visualstudio.com/DefaultCollection/{team-project-name}/_git/repo1

Great.  That brings down the code for Repo1.

Next, create a submodule reference from Repo1 to Repo2.

git submodule add https://{projectname}.visualstudio.com/DefaultCollection/{team-project-name}/_git/repo2

You’ve now got the code from Repo2 in a folder off the root of Repo1.  You also get a new file named ‘.gitmodules’.  That .gitmodules file contains the mapping from Repo1 to Repo2 and this file should be added, committed, and pushed to Repo1 along with any other changes that you make.

The code from Repo1, the repo2 submodule code, and a .gitmodules file

At this point, you’ve got the local workstation stuff working with submodules.  Next, let’s work on the automated build.

TFS Builds with Git Submodules

To make this work with TFS Build vNext, it’s really just a matter of a single checkbox.  In a web browser, go to the build definition and click ‘Edit’ to bring up the build editor.  Click on the Repository tab to view the git repo settings for this build definition.  Towards the bottom of the list, you’ll see a checkbox option that says “Checkout submodules”.  Check that and hit save.  That’s it.  When your build runs, the Git checkout step will follow the submodules that you defined in the .gitmodules file and put all that code on disk.  After that, you can do whatever you need to do with that code from the second repository.

On the repository tab of the build definition, check ‘Checkout submodules’

I hope this helped.

-Ben

 

— Git gettin’ ya down?  Trying to get your Git repos into your DevOps pipelines?  Not even sure what a ‘DevOps’ is?  We can help.  Drop us a line at info@benday.com.  

SUBSCRIBE TO THE BLOG


4 responses to “One TFS Build, Multiple Git Repositories with Submodules”

  1. Thierry Prost Avatar
    Thierry Prost

    Hi Ben, any way that git submodule update –remote can be ran at the beginning of each build?
    Thanks in advance

  2. redcloud319 Avatar
    redcloud319

    agreed! i would like to have the submodule do a fetch before it builds so it has the most up-to-date code from the submodule. Check this link out, Visual Studio Dev community is voting for features. Go vote https://visualstudio.uservoice.com/forums/121579-visual-studio-ide?filter=top&page=2

  3. Ahsen Baig Avatar

    Hello,
    One of your images is broken pointing to a local file path. (“file:///C:/Users/benda/AppData/Local/Temp/SNAGHTML27e793.PNG”)

    Thanks,
    Ahsen Baig

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.