Azure DevOps, Scrum, & .NET Software Leadership and Consulting Services

Free course! Predicting the Future, Estimating, and Running Your Projects with Flow Metrics

Deploy Entity Framework Core 2.1 Migrations from a DLL


We’ve got a new version of .NET Core and a new version of Entity Framework Core (EF Core).  Unfortunately, EF Core 2.1 hasn’t made it any easier to deploy database migrations from DLLs rather than from the source code.  (sigh)  Deploying EF Core migrations from DLLs is a key feature for DevOps CI/CD stuff — especially if you’re doing “The Right Thing” and splitting your builds from your releases.  For example, using VSTS Builds to compile your code and then VSTS Releases to deploy your apps into one or more environments.

Thankfully, there are options.  If you’re trying to deploy EF migrations from TFS or VSTS, you can use my Build & Release Tools extension.  If you’re using something else, I’ve got a batch script for you.  I’ve been maintaining both of these for a while and with this release, I’ve taken some suggestions from the community and added some new features.

Option #1: TFS / VSTS Extension

It’s free.  Go download and install this extension.  After you’ve installed that extension, you can add the Deploy Entity Framework Core Migrations step to your build definition or release definition.

The updated version that I pushed out on 7/4/2018 now give you the option to split your Entity Framework code & migrations DLL from the startup project that you’ll use to deploy your migrations.  For example, if you’ve got an ASP.NET Core web project and then also an API project that has your EF stuff in it, the new version of the extension now handles this.

In order to enable this, the extension now requires that you provide the class name of your DbContext.  Don’t worry…you don’t have to provide the full namespace.  You just have to provide the class name.  This new feature also lets you have multiple DbContext classes in your code and each of those DbContexts can have their own migrations.

To support how all the Entity Framework dependencies are now handled in .NET Core, you’ll need to provide the name of your runtimeconfig.json file for your deployed application.  If you don’t do this, you’ll run into errors where your EF Core dependencies won’t properly resolve.

Option #2: Batch Script

I originally wrote a batch script for EF Core v1 and included it in a blog post and then updated it for EF Core 2.0. This script does almost exactly the same thing as what “dotnet ef database update” does except that it references your already compiled DLLs instead of your source code.

One of the hardest parts of scripting this is trying to figure out where ef.dll is on disk.  The script tries to find ef.dll in a couple different places but if it can’t find it, the script will fail.  If you don’t want to deal with that uncertainty, you can now just manually put ef.dll into the same directory as the script and the script will use that copy.

There are a handful of minor changes to the args for this new version of the script.  The first argument is now the filename for your migrations DLL (it used to be just the namespace).  There are now a couple of optional args.  Argument #2 is the startup DLL filename and argument #3 is your DbContext class name.

deploy-ef2dot1-migrations.bat migrationsDllName [startupDllName] [dbContextClassName]

The startup DLL filename arg is handy for that multi-DLL scenario that I discussed above.  The DbContext class name arg is helpful for when you have multiple DbContext classes in your project.

Here’s a link to download the EF Core 2.1 migration deploy script.

Anyway, I hope this helps.

-Ben

 

— Looking for help with Entity Framework Core?  Need some assistance getting EF Core working with your team’s DevOps pipelines?  Want someone to come help your team figure out why integrating, building your code, and deploying your apps is so difficult and make it a whole lot less difficult?  We can help.  Drop us a line at info@benday.com.   

 

SUBSCRIBE TO THE BLOG


35 responses to “Deploy Entity Framework Core 2.1 Migrations from a DLL”

  1. […] (UPDATE 7/5/2018: If you’re using EF Core 2.1, you’ll probably want to read the updated version of this post.) […]

  2. toralux Avatar

    Thanks for updating the VSTS Extension to support Entity Framework Core 2.1. Anyway, I have still problems using it, because before it picked up the connectionstring from appsettings.json, but now it seams like it picks up appsettings.Development.json (and I don’t have same users defined in my local development environment and on the server where I’m deploying the application). Any ideas?

    1. Ben Day Avatar

      It’s almost definitely because of a rogue environment variable. Go look for “ASPNETCORE_ENVIRONMENT” on that system. It’s probably set to ‘development’ and that’s what’s triggering the load of appsettings.development.json.

      1. toralux Avatar

        The log says: “Using environment ‘Development’.”
        I have not set any environment-variable “ASPNETCORE_ENVIRONMENT=Development” on this system. The VSTS Extension used to work until Entity Framework Core 2.0, and I did no changes to any environment-variables.
        Maybe Development is default environment? Is it possible to override in the VSTS Extension?

        1. Ben Day Avatar

          Hmmm. Something somewhere in the code is loading the development version.

          For some hints on what to look for, go check out the following class in my GitHub repo.

          https://github.com/benday/asp-mvc-invoice-sample/blob/master/src/Benday.InvoiceApp.Api/InvoiceDesignTimeDbContextFactory.cs

          Look for stuff that’s going after environment variables like the CreateDbContext(string[] args) method at line 27. Then look for code that works with ConfigurationBuilder and calls AddJsonFile() like what I’m doing in the Create(string, string) method at line 30.

      2. toralux Avatar

        I’m still not sure where ASPNETCORE_ENVIRONMENT i set. Anyway solved my problem by overriding environment in DbContextFactory by reading it from a file:

        var environment = Environment.GetEnvironmentVariable(“ASPNETCORE_ENVIRONMENT”);
        Console.WriteLine($”Environment from EnvironmentVariable ASPNETCORE_ENVIRONMENT: {environment}”);

        var path = Directory.GetCurrentDirectory();
        Console.WriteLine($”Loading environment.json from: {path}”);

        var config = new ConfigurationBuilder()
        .SetBasePath(path)
        // Read “environment.json” created in powershell during VSTS-release
        .AddJsonFile(“environment.json”, optional: true, reloadOnChange: true);

        // Build an initial configuration
        var conf = config.Build();
        // …and set the environmentname if it exists
        if (conf.GetSection(“ActiveEnvironment”).Exists())
        {
        environment = conf.GetSection(“ActiveEnvironment”).Value;
        Console.WriteLine($”Environment from environment.json ActiveEnvironment: {environment}”);
        }

        And then I have a Powershell VSTS step before running Deploy EF Core migrations with Inline Script:

        param([string]$environmentName, [string]$environmentJsonPath)

        $json = @{
        ActiveEnvironment= $environmentName
        }
        $json | ConvertTo-Json | Set-Content $environmentJsonPath

  3. paulrreynolds Avatar
    paulrreynolds

    Thanks Ben, this is working nicely to generate migration scripts for production, and I can call into this from my CI pipeline.

    One question I have is: Is there a way to copy the correct version of EF.dll into the publish output? I’m using separate servers for Build and Deployment and I’d like to guarantee that whichever EF version was used during the build is the one which is used to generate the migration script during deployment.

    1. Ben Day Avatar

      Hey Paul —

      Sadly, there isn’t a good way to figure that out. If you’re using VSTS/TFS, you can use my Build & Release Tools that are published on marketplace.visualstudio.com. I package ef.dll in that so you’d essentially be getting what you’re asking for without having to work too hard to get it.

      -Ben

  4. Jeremy Holovacs Avatar

    How can I specify a connection string variable to use here? My build has my local dev connection string in it, and none of my other connection strings (UAT, PROD) are in source control or the build by design.

    1. Ben Day Avatar

      Hey Jeremy –

      Where do your other connection strings live? The migration deployment script doesn’t have anything to do with connection strings because it delegates all of that to EF. So whatever EF is going to use to run your migrations is what you’ll need to set.

      Make sense?

      -Ben

  5. Jeremy Holovacs Avatar

    I have the same problem; it says “Using environment ‘Development’”

  6. […] Deploy Entity Framework Core 2.1 Migrations from a DLL Ben Day recommends that you deploy EF Core migrations from DLLs, as a best practice of DevOps deployments – but that’s not always easy. Fortunately, he’s got guidance for you that explains how to do it and a Build & Release Tools extension for VSTS to help you even more. […]

  7. Marcel Beeker Avatar
    Marcel Beeker

    Hi Benjamin,
    Thank you for your extensions. I saves me a lot of work. I’m wondering why Microsoft hasn’t this feature build in VSTS.

    Thanks,
    Marcel

  8. Per Erik Gransøe Avatar

    Is a corresponding bash shell script available osx/linux users?

  9. […] If you’re deploying from a Team Foundation Server (TFS) Build or Release or you’re trying to deploy from an Azure DevOps build or release pipeline, you can use my free build/release extension.  For more information about the extension, check out this blog post. […]

  10. Alexander Shabunevich Avatar

    Ben, first of all thanks for your batch script! It helped me a lot.

    One note, in your script invalid argument was passed in “–startup-assembly .\%EfMigrationsDllName%”. It should be “–startup-assembly .\%StartupDllName%”.

  11. Robert B Avatar
    Robert B

    Your ADO task appears to be using the appsettings.Development.json. Is there a way to stop that? that makes no sense at all that it would use that file instead of appsettings.json.

  12. Robert B Avatar
    Robert B

    OK. Figured it out. It definitely wasn’t code as mentioned by Ben above. I tried many ways to set the environment variable for ASPNETCORE_ENVIRONMENT. After exhausting everything I saw online, I decided to set a variable Release Pipeline for ASPNETCORE_ENVIRONMENT to “Production” and assign it to the Deploy stage. It worked.

    Hopefully this helps someone out.

    1. Ben Day Avatar

      Glad you figured it out! That’s pretty much exactly what I was going to tell you.

      Somewhere in your code, you’ve got something that looks like this

      var builder = new Microsoft.Extensions.Configuration.ConfigurationBuilder()
      .SetBasePath(basePath)
      .AddJsonFile(“appsettings.json”)
      .AddJsonFile($”appsettings.{environmentName}.json”, true)
      .AddEnvironmentVariables();

      It’s that second AddJsonFile() that goes after the dev version of appsettings. You can pretty safely nuke that code or that version of appsettings.json.

      -Ben

  13. Geoff Gray Avatar
    Geoff Gray

    Thanks for the utility Ben. I am trying to implement it in my staging environment, but am running into an error stating it cannot find the SQL connection string. We do not keep any connection strings in our appsettings files. We use secrets files locally and have the connection strings configured in the deployment slots. I can get the connnection string from the Azure Keyvault with powershell, but I need to know where to set it so that the DLL will find it.

    2018-12-11T21:32:09.4267355Z Finding application service provider…
    2018-12-11T21:32:09.4279190Z Finding IWebHost accessor…
    2018-12-11T21:32:09.4279479Z Using environment ‘Development’.
    2018-12-11T21:32:09.4279653Z Using application service provider from IWebHost accessor on ‘Program’.
    2018-12-11T21:32:11.2157487Z System.ArgumentNullException: Value cannot be null.
    2018-12-11T21:32:11.2158512Z Parameter name: connectionString
    2018-12-11T21:32:11.2159367Z at Microsoft.EntityFrameworkCore.Utilities.Check.NotEmpty(String value, String parameterName)
    2018-12-11T21:32:11.2159618Z at Microsoft.EntityFrameworkCore.SqlServerDbContextOptionsExtensions.UseSqlServer(DbContextOptionsBuilder optionsBuilder, String connectionString, Action`1 sqlServerOptionsAction)

  14. Tony Mayes Avatar
    Tony Mayes

    Hey Ben, I’m trying to figure out the best practice on where this goes in the Azure DevOps pipeline. I currently have a build definition that’s dropping a .zip of my published application to the staging directory, that gets copied and the script gets copied, and a release that picks up that drop and uses an Azure App Service Deploy task to deploy the zip, but the script fails because it’s trying to reference the published code, which is all zipped up and in a different directory on the drop. I would like to run this script in the release rather than the build (right?), but maybe I should just do it during the build…seems like more of a release thing.

    Anyway, the issue is that the published zip file and script get copied separately and that zip not unzipped. It would seem to me I’d need to unzip it, drop this script in the bin where the dll is, and it would work. Is that how you intend this to work?

    Thanks, -Tony

  15. Tony Mayes Avatar
    Tony Mayes

    I ended up having luck extracting the files. I think I’m good to go. However, I wanted to note for others that may use this there are lots of directory pathing issues that come up in VSTS during release. I would advise people to use this line instead of the current line 134 (modify line 142 to match), to fix pathing so that it’s always the script’s directory:

    dotnet exec –depsfile “%~dp0\%EfMigrationsDllDepsJson%” –additionalprobingpath %PathToNuGetPackages% –additionalprobingpath %PathToNuGetPackages_Fallback1% –additionalprobingpath %PathToNuGetPackages_Fallback2% –runtimeconfig “%~dp0\%EfMigrationsDllRuntimeConfig%” %PathToEfDll% database update –assembly “%~dp0\%EfMigrationsDllName%” –startup-assembly “%~dp0\%EfMigrationsDllName%” –project-dir “%~dp0” –verbose –root-namespace %EfMigrationsNamespace%

  16. Eric Avatar
    Eric

    It looks like the batch script is not actually using the StartupDllName at all.

  17. Simon Ottaway Avatar
    Simon Ottaway

    Hi Ben,

    I get the following error when I use your migration tool in my Release pipeline:

    2019-06-11T15:36:35.7181615Z An assembly specified in the application dependencies manifest (EFMigrationData.deps.json) was not found:
    2019-06-11T15:36:35.7181819Z package: ‘Microsoft.EntityFrameworkCore.Abstractions’, version: ‘2.2.4’
    2019-06-11T15:36:35.7181992Z path: ‘lib/netstandard2.0/Microsoft.EntityFrameworkCore.Abstractions.dll’
    2019-06-11T15:36:35.7291458Z ##[error]Something unexpected and bad happened. Do you have .NET Core installed on this build agent?
    2019-06-11T15:36:35.7300122Z ##[error]C:\Program Files\dotnet\dotnet.exe failed with return code: 2147516556

    Am I missing a step somewhere? The Build Artifact contains only my Dlls

  18. Filip Van Bouwel Avatar

    Has this been updated for .NET Core 3.1?
    I get the following error after my update from .NET Core 2.2 tot 3.1 :

    Microsoft.EntityFrameworkCore.Tools.CommandException: Your startup project ‘blablabla’ doesn’t reference Microsoft.EntityFrameworkCore.Design. This package is required for the Entity Framework Core Tools to work. Ensure your startup project is correct, install the package, and try again.
    2019-12-30T18:55:05.2875335Z —> System.IO.FileNotFoundException: Could not load file or assembly ‘Microsoft.EntityFrameworkCore.Design, Culture=neutral, PublicKeyToken=null’. The system cannot find the file specified.

    I have referenced version 3.1.0 of Microsoft.EntityFrameworkCore.Design through Nuget in my startup project.

    Thanks for the good work on this build task. I hope it’ll be around for a long time. 🙂

  19. Paul Reynolds Avatar
    Paul Reynolds

    Filip, did you get this to work in .Net Core 3.1? I’m getting the error:
    No project was found. Change the current working directory or use the –project option.

  20. Filip Van Bouwel Avatar

    No, I didn’t. I switched to generating scripts during build and executing those scripts during release. It gives me a bit more control and it’s easier to check the scripts that will be executed.

    1. Paul Reynolds Avatar
      Paul Reynolds

      Yeah, I think I’ll need to switch to that approach as well. I do need to generate the scripts on each deployment though, since we’re deploying the same build to many places at potentially different migration states

    2. Muiz Atolagbe Avatar
      Muiz Atolagbe

      Hi Filip, I am also having issues using the “Deploy EF Core Migrations” task. Please, what is the name of the script generator you used in your release pipeline?

  21. Filip Van Bouwel Avatar

    You can still generate them during build. They have all the checks included to see which migrations have already been run and they’re idempotent, so you can run them multiple times in a row.

  22. danmiser Avatar

    Thanks for the script. Works like a charm and gets us what we need. Well done.

  23. azelcorpuz Avatar
    azelcorpuz

    I manage to make it run with dotnet core 3.1, but i am not using his latest script as provided (i was looking on his old blog), but i saw it was almost the same, the only difference is he is resolving the DLL file, and also if the start up is different from the target:

    set EfMigrationsNamespace=%1
    set EfMigrationsDllName=%1.dll
    set EfMigrationsDllDepsJson=%1.deps.json
    set EfMigrationsRuntimeConfig=%1.runtimeconfig.json
    set DllDir=%cd%
    set PathToNuGetPackages=
    set PathToEfDll=%PathToNuGetPackages%\microsoft.entityframeworkcore.tools\3.1.4\tools\netcoreapp2.0\any\ef.dll

    dotnet exec –depsfile .\%EfMigrationsDllDepsJson% –runtimeconfig .\%EfMigrationsRuntimeConfig% –additionalprobingpath %PathToNuGetPackages% %PathToEfDll% database update –context %2 –assembly .\%EfMigrationsDllName% –startup-assembly .\%EfMigrationsDllName% –project-dir . –data-dir %DllDir% –verbose –root-namespace %EfMigrationsNamespace%

    Though, i needed to published my dotnet core startup project as standalone, or else it is trying to look for some dll.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.