AWS Elastic Beanstalk environment variables in ASP.NET Core 1.0

Question

How do I get environment variables from elastic beanstalk into an asp.net core mvc application? I have added a .ebextensions folder with app.config file in it with the following:

option_settings:
- option_name: HelloWorld
  value: placeholder

- option_name: ASPNETCORE_ENVIRONMENT
  value: placeholder

The .ebextensions folder is included in the publish package.

On deployment, both the variables are visible in the aws elasticbeanstalk console at Configuration > Software Configuration > Environment Variables

However, when I try to read the variables in the application, none of the below options are working:

Environment.GetEnvironmentVariable("HelloWorld") // In controller
Configuration["HelloWorld"] // In startup.cs

Any ideas on what I could be missing? Thanks.

Answer 1

I just implemented a slightly other solution which injects the beanstalk environment variables to the program so that you may access them by Environment.GetEnvironmentVariable():

private static void SetEbConfig()
{
    var tempConfigBuilder = new ConfigurationBuilder();

    tempConfigBuilder.AddJsonFile(
        @"C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration",
        optional: true,
        reloadOnChange: true
    );

    var configuration = tempConfigBuilder.Build();

    var ebEnv =
        configuration.GetSection("iis:env")
            .GetChildren()
            .Select(pair => pair.Value.Split(new[] { '=' }, 2))
            .ToDictionary(keypair => keypair[0], keypair => keypair[1]);

    foreach (var keyVal in ebEnv)
    {
        Environment.SetEnvironmentVariable(keyVal.Key, keyVal.Value);
    }
}

Simply call SetEbConfig(); before building your webhost. With this solution, also AWS SDK does read it's settings like AWS_ACCESS_KEY_ID correctly.

Answer 2

Had the same problem, and just received a reply from AWS support about this issue. Apparently environment variables are not properly injected into ASP.NET Core applications in elastic beanstalk.

As far as I know, they're working to fix the problem.

The workaround is to parse C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration into the configuration builder. This file is part of your elastic beanstalk environment and should be accessible upon deploying your project.

First add the file:

var builder = new ConfigurationBuilder()
    .SetBasePath("C:\\Program Files\\Amazon\\ElasticBeanstalk\\config")
    .AddJsonFile("containerconfiguration", optional: true, reloadOnChange: true);

Then access the values:

var env = Configuration.GetSection("iis:env").GetChildren();

foreach (var envKeyValue in env)
{
    var splitKeyValue = envKeyValue.Value.Split('=');
    var envKey = splitKeyValue[0];
    var envValue = splitKeyValue[1];
    if (envKey == "HelloWorld")
    {
        // use envValue here
    }
}

Courtesy of G.P. from Amazon Web Services

Answer 3

I implemented the other answer to create a convenient workaround to load the environment properties from Elastic Beanstalk directly into your ASP.NET Core app configuration.

For ASP.NET Core 2.0 - edit your Program.cs

Note that this WebHost build was taken from the source code of WebHostBuilder.CreateDefaultBuilder()

https://github.com/aspnet/MetaPackages/blob/dev/src/Microsoft.AspNetCore/WebHost.cs

using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Server.Kestrel.Core;
using Microsoft.AspNetCore.Server.Kestrel.Core.Internal;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace NightSpotAdm
{
    public class Program
    {
        public static void Main(string[] args)
        {
            BuildWebHost(args).Run();
        }

        public static IWebHost BuildWebHost(string[] args)
        {
            // TEMP CONFIG BUILDER TO GET THE VALUES IN THE ELASTIC BEANSTALK CONFIG
            IConfigurationBuilder tempConfigBuilder = new ConfigurationBuilder();

            tempConfigBuilder.AddJsonFile(
                @"C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration",
                optional: true,
                reloadOnChange: true
            );

            IConfigurationRoot tempConfig = tempConfigBuilder.Build();

            Dictionary<string, string> ebConfig = ElasticBeanstalk.GetConfig(tempConfig);

            // START WEB HOST BUILDER
            IWebHostBuilder builder = new WebHostBuilder()
                .UseKestrel()
                .UseContentRoot(Directory.GetCurrentDirectory());

            // CHECK IF EBCONFIG HAS ENVIRONMENT KEY IN IT
            // IF SO THEN CHANGE THE BUILDERS ENVIRONMENT
            const string envKey = "ASPNETCORE_ENVIRONMENT";

            if (ebConfig.ContainsKey(envKey))
            {
                string ebEnvironment = ebConfig[envKey];
                builder.UseEnvironment(ebEnvironment);
            }

            // CONTINUE WITH WEB HOST BUILDER AS NORMAL
            builder.ConfigureAppConfiguration((hostingContext, config) =>
                {
                    IHostingEnvironment env = hostingContext.HostingEnvironment;

                    // ADD THE ELASTIC BEANSTALK CONFIG DICTIONARY
                    config.AddJsonFile(
                            "appsettings.json",
                            optional: true,
                            reloadOnChange: true
                        )
                        .AddJsonFile(
                            $"appsettings.{env.EnvironmentName}.json",
                            optional: true,
                            reloadOnChange: true
                        )
                        .AddInMemoryCollection(ebConfig);

                    if (env.IsDevelopment())
                    {
                        Assembly appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
                        if (appAssembly != null)
                        {
                            config.AddUserSecrets(appAssembly, optional: true);
                        }
                    }

                    config.AddEnvironmentVariables();

                    if (args != null)
                    {
                        config.AddCommandLine(args);
                    }
                })
                .ConfigureLogging((hostingContext, logging) =>
                {
                    logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));
                    logging.AddConsole();
                    logging.AddDebug();
                })
                .UseIISIntegration()
                .UseDefaultServiceProvider(
                    (context, options) => { options.ValidateScopes = context.HostingEnvironment.IsDevelopment(); })
                .ConfigureServices(
                    services =>
                    {
                        services.AddTransient<IConfigureOptions<KestrelServerOptions>, KestrelServerOptionsSetup>();
                    });

            return builder.UseStartup<Startup>().Build();
        }
    }

    public static class ElasticBeanstalk
    {
        public static Dictionary<string, string> GetConfig(IConfiguration configuration)
        {
            return
                configuration.GetSection("iis:env")
                    .GetChildren()
                    .Select(pair => pair.Value.Split(new[] { '=' }, 2))
                    .ToDictionary(keypair => keypair[0], keypair => keypair[1]);
        }
    }
}

For ASP.NET Core 1.0

    public Startup(IHostingEnvironment env)
    {
        var builder = new ConfigurationBuilder()
            .SetBasePath(env.ContentRootPath)
            .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
            .AddJsonFile(@"C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration", optional: true, reloadOnChange: true)
            .AddEnvironmentVariables();

        var config = builder.Build();

        builder.AddInMemoryCollection(GetEbConfig(config));

        Configuration = builder.Build();
    }

    private static Dictionary<string, string> GetEbConfig(IConfiguration configuration)
    {
        Dictionary<string, string> dict = new Dictionary<string, string>();

        foreach (IConfigurationSection pair in configuration.GetSection("iis:env").GetChildren())
        {
            string[] keypair = pair.Value.Split(new [] {'='}, 2);
            dict.Add(keypair[0], keypair[1]);
        }

        return dict;
    }

Answer 4

Instead of having to parse the containerconfiguration you can leverage the ebextensions options to set the variable as part of your deploy process:

commands:
  set_environment: 
    command: setx ASPNETCORE_ENVIRONMENT "Development" /M

This will set a global environment variable as part of your application deployment. This variable use-case is officially supported and documented by Microsoft.

After deploying your app you can verify the setting is set correctly in the EC2 instance:

Answer 5

AWS addressed this issue in the Elastic Beanstalk Windows Server platform update on June 29, 2020:

Previously, Elastic Beanstalk didn't support passing environment variables to .NET Core applications and multiple-application IIS deployments that use a deployment manifest [1]. The Elastic Beanstalk Windows Server platform update on June 29, 2020 [2] now fixes this gap. For details, see Configuring your .NET environment in the Elastic Beanstalk console [3].

[1] https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/dotnet-manifest.html

[2] https://docs.aws.amazon.com/elasticbeanstalk/latest/relnotes/release-2020-06-29-windows.html

[3] https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_NET.container.console.html#dotnet-console

Answer 6

Above solution doesnt helped me to load config file based on enviroment settings. So here is my solution AWS ElasticBeansTalk "hack"

    public Startup(IHostingEnvironment env)
    {
        var builder = new ConfigurationBuilder()
            .SetBasePath(env.ContentRootPath)
            .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
            .AddJsonFile($"appsettings.{GetEnvVariableAWSBeansTalkHack(env)}.json", optional: true)
            .AddEnvironmentVariables();

        Configuration = builder.Build();
    }

    private static string GetEnvVariableAWSBeansTalkHack(IHostingEnvironment env)
    {
        var config = new ConfigurationBuilder()
           .AddJsonFile(@"C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration", optional: true, reloadOnChange: true).Build();

        Dictionary<string, string> dict = new Dictionary<string, string>();
        foreach (IConfigurationSection pair in config.GetSection("iis:env").GetChildren())
        {
            string[] keypair = pair.Value.Split(new[] { '=' }, 2);
            dict.Add(keypair[0], keypair[1]);
        }

        return dict.ContainsKey("ASPNETCORE_ENVIRONMENT") 
                ? dict["ASPNETCORE_ENVIRONMENT"] 
                : env.EnvironmentName;
    }

Answer 7

You can create an implementation of Microsoft.Extensions.Configuration.

Also available at https://gist.github.com/skarllot/11e94ed8901a9ddabdf05c0e5c08dbc5.

using Microsoft.Extensions.Configuration;
using Newtonsoft.Json.Linq;
using System.IO;
using System.Linq;

namespace Microsoft.Extensions.Configuration.AWS
{
    public class AmazonEBConfigurationProvider : ConfigurationProvider
    {
        private const string ConfigurationFilename = @"C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration";

        public override void Load()
        {
            if (!File.Exists(ConfigurationFilename))
                return;

            string configJson;
            try
            {
                configJson = File.ReadAllText(ConfigurationFilename);
            }
            catch
            {
                return;
            }

            var config = JObject.Parse(configJson);
            var env = (JArray)config["iis"]["env"];

            if (env.Count == 0)
                return;

            foreach (var item in env.Select(i => (string)i))
            {
                int eqIndex = item.IndexOf('=');
                Data[item.Substring(0, eqIndex)] = item.Substring(eqIndex + 1);
            }
        }
    }

    public class AmazonEBConfigurationSource : IConfigurationSource
    {
        public IConfigurationProvider Build(IConfigurationBuilder builder)
        {
            return new AmazonEBConfigurationProvider();
        }
    }

    public static class AmazonEBExtensions
    {
        public static IConfigurationBuilder AddAmazonElasticBeanstalk(this IConfigurationBuilder configurationBuilder)
        {
            configurationBuilder.Add(new AmazonEBConfigurationSource());
            return configurationBuilder;
        }
    }
}

Then use with your ConfigurationBuilder:

var builder = new ConfigurationBuilder()
    .SetBasePath(env.ContentRootPath)
    .AddJsonFile("appsettings.json", true, true)
    .AddJsonFile($"appsettings.{env.EnvironmentName}.json", true)
    .AddAmazonElasticBeanstalk()    // <-- Merge with other sources
    .AddEnvironmentVariables();

Answer 8

.NET Core 2 + posrgresql RDS

Further to @sebastian's great answer above, I found that the settings were in a different part of the file, viz. plugins:rds:env.

Also there was no need to split on =, so the parsing code I have is:

private static void SetEbConfig()
        {
            var tempConfigBuilder = new ConfigurationBuilder();

            tempConfigBuilder.AddJsonFile(
                @"C:\Program Files\Amazon\ElasticBeanstalk\config\containerconfiguration",
                optional: true,
                reloadOnChange: true
            );

            var configuration = tempConfigBuilder.Build();

            var ebEnv = configuration.GetSection("plugins:rds:env")
                                        .GetChildren()
                                        .ToDictionary(child => child.Key, child => child.Value);

            foreach (var keyVal in ebEnv)
            {
                Environment.SetEnvironmentVariable(keyVal.Key, keyVal.Value);
            }
        }

The relevant (and redacted ;-)) JSON is as follows:

{
    "plugins": {
        "rds": {
            "Description": "RDS Environment variables",
            "env": {
                "RDS_PORT": "....",
                "RDS_HOSTNAME": "....",
                "RDS_USERNAME": "....",
                "RDS_DB_NAME": "....",
                "RDS_PASSWORD": "...."
            }
        }
    }
}

(This reply is separate since I don't have rep to comment...)

Answer 9

This can definitely be done in an .ebextensions folder. Simply create a new file in your .ebextensions folder (I used a name of "options.config"), mark it as "copy if newer" or "copy always" and make sure you use the option_settings header with a aws:elasticbeanstalk:application:environment namespace:

option_settings:
  aws:elasticbeanstalk:application:environment:
    MyEnvVar: SomeValue

EDIT: I forgot to include a link to the docs! https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-softwaresettings.html

Answer 10

Update for ASP.net Core 3

Set ASPNETCORE_ENVIRONMENT environment variable in Elastic Beanstalk to Staging, Production, Development...

In Core project create appsettings.Staging.json and this configuration will be used when this the project is deployed.