Scott Hanselman

Best practices for private config data and connection strings in configuration in ASP.NET and Azure

January 6, '16 Comments [50] Posted in ASP.NET
Sponsored By

Image Copyright Shea Parikh / getcolorstock.com - used under licenseA reader emailed asking how to avoid accidentally checking in passwords and other sensitive data into GitHub or source control in general. I think it's fair to say that we've all done this once or twice - it's a rite of passage for developers old and new.

The simplest way to avoid checking in passwords and/or connection strings into source control is to (no joke) keep passwords and connection strings out of your source.

Sounds condescending or funny, but it's not, it's true. You can't check in what doesn't exist on disk.

That said, sometimes you just need to mark a file as "ignored," meaning it's not under source control. For some systems that involves externalizing configuration values that may be in shared config files with a bunch of non-sensitive config data.

ASP.NET 4.6 secrets and connection strings

Just to be clear, how "secret" something is is up to you. If it's truly cryptographically secret or something like a private key, you should be looking at data protection systems or a Key Vault like Azure Key Vault. Here we are talking about medium business impact web apps with API keys for 3rd party web APIs and connection strings that can live in memory for short periods. Be smart.

ASP.NET 4.6 has web.config XML files like this with name/value pairs.

<appSettings>      
<add key="name" value="someValue" />
<add key="name" value="someSECRETValue" />
</appSettings>

We don't want secrets in there! Instead, move them out like this:

<appSettings file="Web.SECRETS.config">      
<add key="name" value="someValue" />
</appSettings>

Then you just put another appSettings section in that web.secrets.config file and it gets merged at runtime.

NOTE: It's worth pointing out that the AppSettings technique also works for Console apps with an app.config.

Finally, be sure to add Web.secrets.config (or, even better, make it *.secrets and use a unique extension to identify your sensitive config.

This externalizing of config also works with the <connectionStrings> section, except you use the configSource attribute like this:

<connectionStrings configSource="secretConnectionStrings.config">
</connectionStrings>

Connection Strings/App Secrets in Azure

When you're deploying a web app to Azure (as often these apps are deployed from source/GitHub, etc) you should NEVER put your connection strings or appSettings in web.config or hard code them.

Instead, always use the Application Settings configuration section of Web Apps in Azure.

Application Settings and Secrets in Azure

These collection strings and name value pairs will automatically be made available transparently to your website so you don't need to change any ASP.NET code. Considered them to have more narrow scope than what's in web.config, and the system will merge the set automatically.

Additionally they are made available as Environment Variables, so you can Environment.GetEnvironmentVariable("APPSETTING_yourkey") as well. This works in any web framework, not just ASP.NET, so in PHP you just getenv('APPSETTING_yourkey") as you like.

The full list of database connection string types and the prepended string used for environment variables is below:

  • If you select “Sql Databases”, the prepended string is “SQLAZURECONNSTR_”
  • If you select “SQL Server” the prepended string is “SQLCONNSTR_”
  • If you select “MySQL” the prepended string is “MYSQLCONNSTR_”
  • If you select “Custom” the prepended string is “CUSTOMCONNSTR_”

ASP.NET 5

ASP.NET 5 has the concept of User Secrets or User-Level Secrets where the key/value pair does exist in a file BUT that file isn't in your project folder, it's stored in your OS user profile folder. That way there's no chance it'll get checked into source control. There's a secret manager (it's all beta so expect it to change) where you can set name/value pairs.

ASP.NET also has very flexible scoping rules in code. You can have an appSettings, then an environment-specific (dev, test, staging, prod) appSettings, then User Secrets, and then environment variables. All of this is done via code configuration and is, as I mentioned, deeply flexible. If you don't like it, you can change it.

var builder = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);

if (env.IsDevelopment())
{
// For more details on using the user secret store see http://go.microsoft.com/fwlink/?LinkID=532709
builder.AddUserSecrets();
}

builder.AddEnvironmentVariables();
Configuration = builder.Build();

So, in conclusion:

  • Don't put private stuff in code.
    • Seems obvious, but...
  • Avoid putting private stuff in common config files
    • Externalize them AND ignore the externalized file so they don't get checked in
  • Consider using Environment Variables or User-level config options.
    • Keep sensitive config out of your project folder at development time

I'm sure I missed something. What are YOUR tips, Dear Reader?

Resources

Image Copyright Shea Parikh - used under license from http://getcolorstock.com


Sponsor: Big thanks to Infragistics for sponsoring the blog this week! Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls.  Get super-charged performance with the world’s fastest HTML5 Grid - Download for free now!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Wednesday, 06 January 2016 07:20:14 UTC
So your saying that storing a connection string on the server in web.config is bad?
Mike
Wednesday, 06 January 2016 08:51:20 UTC
Only if it contains sensitive information i would assume...
Thomas
Wednesday, 06 January 2016 08:54:09 UTC
In previous ASP.NET versions you also did not store the secret file in the project folder but somewhere outside like a user directory which never gets checked into source control. The configSource attribute can point to any location.

@Mike: yes don't store your environment or secret data in web.config. This way you never have to transform your config when you promote your builds.
Wednesday, 06 January 2016 08:57:19 UTC
If you do add secrets into any web.config file (Even a web.secrets.config file as in the example Scott gave), don't forget to encrypt it. More info here.
Wednesday, 06 January 2016 09:13:35 UTC
He's saying that you should keep your secrets (user names, passwords, connection strings, etc.) hidden. Which means not checking it into source control (tfs, github, svn, etc.).

Of course you will need them in test or prod and everything in between. But those secrets should live on the server, which is hopefully better secured than your source control. So you set them when you deploy to the server (only the first time, then just don't override the settings unless you really, really need to). That way, if you accidentally create a public instead of a private github repo, no bad guy can see those secrets.

You'll need access during development, of course. But those configs should be passed around through a different way than storing them generally in your source control. Maybe even a different access code for each developer. That way you can monitor who does what.
KenBonny
Wednesday, 06 January 2016 09:52:09 UTC
I think you ought to mention that using "configSource" doesn't allow for merging of the values contained in both web.config and the external file.

http://stackoverflow.com/questions/6940004/asp-net-web-config-configsource-vs-file-attributes
Pauli Østerø
Wednesday, 06 January 2016 10:00:25 UTC
One of things we have done in the past to avoid this issue is to have environment specific (eg. UAT, PROD) configuration files outside the source code in a separate repository (NOT in source control). No developer and only the IT guy has access to the repository.

We then use combination of PowerShell and TeamCity tasks to replace the configuration file at the time of deploying a package. The team city had two builds: One to just build the solution, other to deploy it to specific environment.

The advantage to this approach is that we always have one single build which propagates from SIT to UAT and PROD.
Ankit Vijay
Wednesday, 06 January 2016 10:01:26 UTC
This was actually a very annoying problem in our team and my question on stackoverflow got no useful answer.
Thanks to you, now we know how to avoid this problem.
Alireza
Wednesday, 06 January 2016 11:56:14 UTC
I find this useful to share connection strings and settings in projects as well. Link - not add - the files to other projects, and then you only need to make changes in one place. For non-web projects, I've learned you need to set the Copy to Output Directory to Copy always to ensure the files get copied to the bin folder on build. For web projects, I haven't figured out how to get IIS Express to find the config files unless they actually exist on disk. Symlinks can work but can be awkward. For me it's been easier and simpler to keep the config files in the web project, make sure they're ignored and then link to those files in the other projects that need them. As stated above, I've also found this can work better than web.config transforms. I have yet to find a setting in the web.config that needs to change between environments and that I can't externalize.
Tim
Wednesday, 06 January 2016 12:28:19 UTC
What's the recommended way to encrypt that data?
Wednesday, 06 January 2016 13:46:23 UTC
For encrypting web.config, I use aspnet_regiis. This probably isn't a solution for keeping passwords out of source control, but it is useful to prevent plain text passwords in deployed code. I use it often for vendor delivered applications.

# Encrypt the appSettings section of web.config
& $env:WinDir\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -pef "appSettings" "C:\inetpub\wwwroot"

# If later you need to decrypt that section, you can run
& $env:WinDir\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -pdf "appSettings" "C:\inetpub\wwwroot"


ASP.NET IIS Registration Tool (Aspnet_regiis.exe)
Elijah W. Gagne
Wednesday, 06 January 2016 13:51:47 UTC
Is something missing from the statement:

"Finally, be sure to add Web.secrets.config (or, even better, make it *.secrets and use a unique extension to identify your sensitive config."
Kerry Jenkins
Wednesday, 06 January 2016 14:04:55 UTC
Needful Information.
Great discussion.
Mahtab Husain Abdi
Wednesday, 06 January 2016 14:55:47 UTC
The P&P for securely encrypting configuration has been the configProtectionProvider for a long time. This makes on-premises deployment to multiple servers much simpler through TFS builds.

I was wondering is the support for configProtectionProvider being dropped entirely with the json file format?
Joseph
Wednesday, 06 January 2016 14:58:13 UTC
@Kerry Jenkins - if you do that make sure that your webserver doesn't serve those files directly: By leaving it with a .config extension IIS will refuse requests for it by default, and your teams editors will likely help with the file syntax.
Wednesday, 06 January 2016 15:56:25 UTC
It's worth mentioning that if you're an Azure user the Azure Key Vault is another option for safely storing secrets outside your application.
Wednesday, 06 January 2016 16:01:13 UTC
Someone is revisiting basics after long time. As always very informative.
Sumit
Wednesday, 06 January 2016 17:15:48 UTC
I'm wondering about the whole "secret in web.config" and encryption. First off, encryption in web.config was ugly every time I've looked at it because it's machine dependent on encryption and decryption and that was a problem with dev/test/prod environments (or maybe I'm doing it wrong). Putting the encryption thing aside the easier thing is to use Azure properties to set your values because that's safer than any other system. If IIS borks and exposes your raw web.config file (although I've never seen that happen) then it won't matter. It begs the question though and makes config transforms somewhat irrelevant. Currently I have dev/test/prod web.config transforms for connection strings, etc. but thinking it might be easier to ditch all those conversions and just simply use placeholder values and let Azure replace the values I need, safe behind my corporate Azure login. It's also a question of what are you protecting this against? If you use dummy values and replace them in Azure then even if the values get exposed you're only looking at stubs. The real protection is on your Azure account and if's the case, then you really only need to leverage that and encryption and extra config files are unnecessary. Okay, I think I've had way too much coffee this morning.
Wednesday, 06 January 2016 18:31:02 UTC
Be careful with .secrets. By default, it can be guessed and downloaded if it is available on the server.

.config files are exempt from downloading.

Good tips, though.
PRMan
Wednesday, 06 January 2016 19:38:39 UTC
Very nice. This will be very useful as I have seen folks (including yours truly) push Twitter API keys and other secrets into our open source repositories. This will allow adding a secrets config file to .gitignore, like you mentioned, and reduce the chance of doing this again.
Wednesday, 06 January 2016 23:31:16 UTC
I'm not sure how it works with Azure, but for classic server deployment, there seems to be no good (i.e. secure, yet easy to manage) out-of-the-box strategy for sensitive data management. When assessing sensitive data protection techniques, it's not enough to just exclude sensitive settings from the source control. You also need to consider how they will be handled during deployment, what the vulnerabilities of the encryption (I assume nobody advocates storing sensitive data in plain text, right/) algorithms are (i.e. what it takes for a hacker to decrypt data once he gets access to a server), how easy it's to share them with other developers during the development phase, and so on.

The most reasonable approach to me is base on the SOD (separation of duties) rules: sensitive settings in development environment are not protected beyond the infrastructure controls (access to source control system, intranet, etc), while sensitive settings in all other environments (QA, pre-prod, prod) are closely guarded by and known to a designated team (like systems engineers assigned to project). So, it's okay to have a connection string in plain text (or encrypted with a known secret) checked in as long as it points to the dev environment. This way it's easy for development team to share code and use common dev servers for dev testing and dev work. Once the code is out for deployment (in QA, pre-prod, prod), then all sensitive settings must be encrypted by a dedicated team members (specifically, non-developers) with the keys/passwords/certs that only they know (using aspnet_regiis, ciphesafe, or whatever tool or technique is approved by the corporate InfoSec policies). This may not be a preferred method for all projects, but it seems to be working better than anything else I've seen in the enterprise environment.

Wednesday, 06 January 2016 23:49:43 UTC
With ASP.NET 5, you can have a custom configuration provider to grab these settings only at runtime from some service. We use this one with Octopus Deploy and plan to have similar ones for Consul and Vault too. Now you need to secure only one secret, which you use to access that service :)
Thursday, 07 January 2016 02:51:58 UTC
@Bil Simser, when you use web.config encryption, you can set the machineKey element in the config to get consistent encryption/decryption of the values. Just use a randomly generated key.

http://www.allkeysgenerator.com/Random/ASP-Net-MachineKey-Generator.aspx
Matt S.
Thursday, 07 January 2016 03:43:12 UTC
Very useful post about security in ASP.NET application. Thanks.
Thursday, 07 January 2016 08:17:22 UTC
I believe that between Azure KeyVault and Azure configuration settings, developers have many options when it comes to securing their config settings and sensitive data. Your suggestions regarding different config sections and merging work really well in dev and other environments (qa, uat) too. In conclusion, if you have sensitive info checked in source control then you're doing it wrong.
Thursday, 07 January 2016 13:08:50 UTC
Encrypting sensitive data (like in web.config) on a server to me "only" protects the data in case someone gets access to the file from the outside, for example, if IIS would deliver a .config file to a client due to a configuration error or bug.

But it does not help anything in case the "outside person" got access to the server/OS or even against inside persons that should not but accidentally have access to the file because of, for example, insufficient ACLs - because in this case the person can easily decrypt it just like the application that uses this file can.

I'm seeing this right, do I?
Bernhard
Thursday, 07 January 2016 14:43:13 UTC
@KerryJenkins I was going to point that out as well. I suspect Scott was going to use that paragraph to reiterate his tip about excluding that file from source control.
Thursday, 07 January 2016 15:44:43 UTC
Cool article! I had this question once, and all I could find was to encrypt it, which I had no idea how to do, haha.
Jeff
Thursday, 07 January 2016 20:35:40 UTC
I came across Git-Secrets the other day, which has some neat hooks and commands to prevent from ever being committed to your repo based on regex matches. This could be a good tool to prevent this entire class of mistake from occurring.
Friday, 08 January 2016 09:22:49 UTC
Hey Scott,
Time to update your footer to say 2016! try not hard coding to avoid updating every year.

Anyways, Happy new year.

P.S I love to read your blogs :)
tina
Friday, 08 January 2016 19:37:32 UTC
[Long time listener, first time caller.]

I went straight to Azure KeyVault and haven't looked back. I store all connection strings, 3rd party API credentials, or anything else somewhat sensitive in KeyVault as a secret.

A little PowerShell gets it created and configured: https://azure.microsoft.com/en-gb/documentation/articles/key-vault-get-started/

A small KeyVault class added to the project template retrieves secrets, using sample code from here: https://github.com/Azure/azure-content/blob/master/articles/key-vault/key-vault-use-from-web-application.md

Then add secret1URL, secret2URL, etc. in the Azure Web App settings which point to the KeyVault URL's for each secret.

Now, there is nothing sensitive in the source code, repo, or Web App config settings.
Todd
Saturday, 09 January 2016 16:22:07 UTC
The problem with aspnet 5 os that you can't get at build configuration in gulp or anywhere else so there is no way of editing values in type/JavaScript for things like api paths and if you're not using mvc you have no way of injecting the values from config into you html/JavaScript.

The build config being chosen needs to be exposed to gulp and other task runners properly. (Real build config like debug, release etc.)
James Hancock
Saturday, 09 January 2016 17:53:57 UTC
For Classic server deployment and Azure there is a simple and reliable solution. My company have used Octopus Deploy for a while now and it has completely changed how we deploy and publish. Octopus Deploy is even free for small teams.

Use the built-in feature to transform the web/app config and know that it will happen every time you deploy a new version.

In the config file in your repository, do have URLs and logins to your dev environment but secure the dev environment do that it is not a big deal if the credentials are known.
Monday, 11 January 2016 00:44:42 UTC
@Bernhard, if your website runs as a low privileges user and the hacker has uploaded a shell to your site, for instance, they will still have to elevate before decrypting your configuration, so it does actually make their work a little more tedious.
Rikard
Monday, 11 January 2016 07:15:32 UTC
Hi Scott, coincidentally I finally managed to publish my blog post Protecting web application secrets in Azure a few days ago. The approach I propose is to put the encrypted setting values into the config files (as usual) and use Azure Key Vault for performing decryption. I like the approach as it provides a better story for configuration management, providing change log etc. for the settings also. Instead of having to keep track of all secrets outside of source control, we just need to keep track of the encryption key (certificate).
Tuesday, 12 January 2016 01:07:48 UTC
Thanks for this. It really helps some of us when someone comes along and just tells us what to do. It at least clears away the noise and gets us started on something.

Sometimes there are simply too many options for us to sift through at the early stages of a project.

Good post.
bill
Tuesday, 12 January 2016 08:22:33 UTC
I thought it is worth sharing here, you can have the same type of configuration from ASP.NET 5 in any other .NET application as well: http://dusted.codes/aspnet-5-like-configuration-in-regular-dotnet-applications.
Tuesday, 12 January 2016 08:23:09 UTC
Sorry, didn't link it properly in my previous comment:
http://dusted.codes/aspnet-5-like-configuration-in-regular-dotnet-applications
Tuesday, 12 January 2016 18:42:43 UTC
The 12-Factor App's approach to configuration encourages the use of environment variables for sensitive (and deploy-specific) config data.
Brian Windheim
Wednesday, 13 January 2016 07:09:06 UTC
At last, in the year 2016, we get some support for deploying different configuration to different environments. For the last 15 years I have been forced to develop this (by using a bunch of different hostile tools) in every single project I've worked in, it has been a PAIN, and something that you expect an eco system (IDE+framework+OS) should take care of 100%. Now, if only all kind of .net apps (console, winform, winservices, etc) would support this infrastructure...
Johnny
Wednesday, 13 January 2016 07:13:46 UTC
How do you version control credentials? YES IT NEEDS TO BE DONE.
Kumar
Wednesday, 13 January 2016 08:03:53 UTC
Back in the old days, we used to be able to encrypt config sections using the Windows DPAPI. What would I use to deploy a secret inside an ASP.Net 5 app on a user's Macbook?
Sean Kearon
Wednesday, 13 January 2016 10:09:35 UTC
Support for external *.config parts isn't new to .Net 4.6, it was there for long.
There's no such thing like ASP.Net 4.6, the author confuses .Net 4.6 and ASP.Net 5.
metz2000
Wednesday, 13 January 2016 10:55:50 UTC
Scott,

I have employed another technique in projects where any sensitive or variable information would be replaced by a placeholder in the CONFIG. Its is this "template" config that we check into the source control.

We then have a utility that can be run against DEV, TST, STAGE, PROD environments that replaces the place holders with values based on the target environment.

The utility only stores non sensitive data in an XML file. Sensitive data like keys, passwords are to be keyed into the utility during deployment.

Comments would be appreciated if this technique can still be refined.

Thank you.

Sundip
Sundip Nair
Wednesday, 13 January 2016 11:06:37 UTC
The article states that it is implied that storing sensitive data in code/configs is necessarily bad. While that may be in some cases, there is no qualification of this principle. If the repo is private and the only people with access are the devs who have the credentials already, why bother adding extra layers of complexity? I believe in the simplest thing that can possibly work (i.e. YAGNI), which in the above case would be just to put the details in the config. I appreciate sometimes this is not the case, but it is not a best practice if it serves no purpose and only adds complexity.
aaron
Wednesday, 13 January 2016 14:33:59 UTC
For development in Classic ASP YEARS ago, we stored the connection string in the registry of a server and developed a COM object that would read only that specific key from the registry into an application level variable at initialization time. This kept connection strings out of the source code, and had an additional bonus in that the connection string was specific to that environment. Application servers for Dev, QA, Staging and production each had their own unique connection string that would point the application to the right database without a single change in code. I think this is still a viable strategy, however the methods used might vary slightly today.
Thirtybird
Wednesday, 13 January 2016 15:35:32 UTC
While this works well for regular connection strings, I have been having issues with getting the Azure connection strings section to work for EF connection strings. Azure doesn't seem to like the format.
Scott Rudy
Tuesday, 19 January 2016 22:42:34 UTC
Yes Scott, very good idea to keep "secret" stuff in files that are on MIME list. Just wait till google.. or hacker find the name and download the config..
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.