Fededim.Extensions.Configuration.Protected implements a custom ConfigurationBuilder and a custom ConfigurationProvider defining a custom tokenization tag which whenever found inside a configuration value decrypts the enclosed encrypted data using ASP.NET Core Data Protection API.
Introduction
Almost one month ago, I posted this article ProtectedJson about an improved ConfigurationSource
and ConfigurationProvider
for JSON files which allowed partial or full encryption of configuration values using Data Protection API. Some comments came through and one which was not so "meaningful" had a question whether my package also supported environment variables.
Even though we could wonder why someone ever wants to encrypt/decrypt environment variables, this question made me have an epiphany: can what I had already done for JSON files also be extended to other configuration sources? After a small proof of concept project after work, the answer was yes and after some rework, I published a new package Fededim.Extensions.Configuration.Protected which is, most of all, an improvement and an extension of ProtectedJson
to support encryption/decryption of configuration values stored inside ANY configuration source.
Key Features
- Encrypt partially or fully a configuration value
- Works with any existing and (hopefully) future
ConfigurationSource
and ConfigurationProvider
(successfully tested with framework's builtin providers like CommandLine, EnvironmentVariables, Json, Xml and InMemoryCollection
) - Transparent in memory decryption of encrypted values without almost any additional line of code
- Supports a global configuration and an eventual custom override for any
ConfigurationSource
- Supports almost any NET framework (net6.0, netstandard2.0 and net462)
- Pluggable easily into any existing NET / NET Core project
- Supports automatic re-decryption on configuration reload if underlying
IConfigurationProvider
supports it
Background
ASP.NET Configuration is the standard .NET Core way of storing application configuration data through hierarchical key-value pairs inside a variety of configuration sources (usually JSON files, but also environment variables, XML file, in memory dictionaries, command line parameters or any custom provider you would like to implement). While .NET Framework used a single source (usually, a XML file which was intrinsically more verbose), .NET Core can use multiple ordered configuration sources, which gets "merged" allowing the concept of overriding of the value of a key in a configuration source with the same one present in a subsequent configuration source. This is useful because in software development, there are usually multiple environments (Development, Integration, PRE-Production and Production) and each environment has its own custom settings (for example, API endpoints, database connection strings, different configuration variables, etc.). In .NET Core, this management is straightforward, in fact, you usually have two JSON files:
- appsettings.json: which contains the configuration parameters common to all environments
- appsettings.<environment name>.json: which contains the configuration parameters specific to the particular environment
ASP.NET Core apps usually configure and launch a host. The host is responsible for app startup, configuring dependency injection and background services, configuring logging, lifetime management and obviously configuring application configuration. This is done mainly in two ways:
- Implicitly, by using one of the framework provided methods like
WebApplication.CreateBuilder
or Host.CreateDefaultBuilder
(usually called inside the Program.cs source file) which substantially do:
- Read and parse command line arguments
- Retrieve environment name respectively from
ASPNETCORE_ENVIRONMENT
and DOTNET_ENVIRONMENT
environment variable (set either in the operating system variables or passed directly in the command line with --environment
argument). - Read and parse two JSON configuration files named appsettings.json and appsettings.<environment name>.json.
- Read and parse the environment variables.
- Call the delegate
Action<Microsoft.Extensions.Hosting.HostBuilderContext,Microsoft.Extensions.Configuration.IConfigurationBuilder>
of ConfigureAppConfiguration
where you can configure the app configuration through IConfigurationBuilder
parameter
- Explicitly by instantiating the
ConfigurationBuilder
class and using one of the provided extensions methods:
AddCommandLine
: to request the parsing of command line parameters (either by --
or -
or /
) AddJsonFile
: to request the parsing of a JSON file specifying whether it is mandatory or optional and whether it should be reloaded automatically whenever it changes on filesystem. AddXmlFile
: to request the parsing of an XML file specifying whether it is mandatory or optional and whether it should be reloaded automatically whenever it changes on filesystem. AddEnvironmentVariables
: to request the parsing of environment variables - etc.
In essence, every Add<xxxx>
extension method adds a ConfigurationSource
to specify the source of key-value pairs (CommandLine, Json File, Environment Variables, etc.) and an associated ConfigurationProvider
used to load and parse the data from the source into the Providers
list of IConfigurationRoot
interface which is returned as a result of the Build
method on ConfigurationBuilder
class as you can see the picture below.
(Inside configuration.Providers
, there are six sources: CommandLineConfigurationProvider
, two JsonConfigurationProvider
for both appsettings.json and appsettings.<environment name>.json, a XMLConfigurationProvider
for the XML file appsettings.xml, a MemoryConfigurationProvider
for the provided dictionary and finally EnvironmentVariableConfigurationProvider
for the environment variables).
As I wrote earlier, the order in which the Add<xxxx> extension methods are called is important because when the IConfigurationRoot
class retrieves a key value, it uses the GetConfiguration method which cycles the Providers
list in a reversed order trying to return the first one which contains the queried key, thus simulating a "merge" of all configuration sources (LIFO order, Last In First Out).
Using the Code
You find all the source code on my Github repository, the code is based on .NET 6.0 and Visual Studio 2022. Inside the solution file, there are four projects, two about the older ProtectedJson
and two about this new Protected
package, let's talk about the last two:
FDM.Extensions.Configuration.Protected
: This is a class library which implements ProtectedConfigurationBuilder
, ProtectedConfigurationProvider
and the extension methods for IConfigurationBuilder
interface (WithProtectedConfigurationOptions
used to specify a particular configuration which applies only to a specific ConfigurationSource
) and for IDataProtect
interface (used to encrypt values from the various configuration sources). FDM.Extensions.Configuration.Protected.ConsoleTest
: This is a console application which shows how to use ProtectedConfigurationBuilder
by reading and parsing six encrypted bespoke configuration sources and converting them to a strongly type class called AppSettings
. The decryption happens flawlessly and automatically without almost any line of code, let's see how.
To use the automatic decryption feature, you have just to replace the call new ConfigurationBuilder()
with a call to new ProtectedConfigurationBuilder()
passing to it the Data Protection API and the custom tokenization tag configuration. After having done that, you can add any existing configuration source by using the standard methods like AddCommandLine
, AddJsonFile
, AddXmlFile
, AddInMemoryCollection
, AddEnvironmentVariables
or even future configuration sources since this package should support all of them as long as the implementation of the GetChildKeys of Microsoft.Extensions.Configuration.ConfigurationProvider does not change (keep reading below to understand the reason). The constructor for ProtectedConfigurationBuilder
takes these parameters:
protectedRegexString
: It is a regular expression string
which specifies the tokenization tag which encloses the encrypted data; it must define a named group called protectedData
. If null
, this parameter assumes the default value:
public const string DefaultProtectedRegexString = "Protected:{(?<protectedData>.+?)}";
The above regular expression essentially searches in a lazy way (so it can retrieve all the occurrences inside a value) for any string
matching the pattern 'Protected:{<encrypted data>}'
and extracts the <encrypted data>
substring storing it inside a group named protectedData
. If you do not like this tokenization, you can replace it with any other one you prefer by crafting a regular expression with the constraint that it extracts the <encrypted data>
substring in a group called protectedData
.
dataProtectionServiceProvider
: This is a IServiceProvider
interface needed to instance the IDataProtectionProvider
of Data Protection API in order to decrypt the data. This parameter is mutually exclusive to the next one. dataProtectionConfigureAction
: This is an Action<IDataProtectionBuilder>
used to configure the Data Protection API in standard NET Core. Again, this parameter is mutually exclusive to the previous one. keyNumber
: used to specify the key index (it will get automatically created, the default value is 1) to use for encryption (Data Protection API supports multiple encryption keys which are derived from the configured master key and strictly connected to a purpose
string passed to the CreateProtector
API, the provided index is used to construct a this purpose
string, for further information please read here and here)
The dataProtectionServiceProvider
and dataProtectionConfigureAction
parameters are somewhat a drawback, because they represent a reconfiguration of another dependency injection for instantiating the IDataProtectionProvider
needed to decrypt the data.
In fact, in a standard NET Core application, usually the dependency injection is configured after having read and parsed the configuration file (so all configuration sources and providers do not use DI), but in this case, I was compelled since the only way to access Data Protection API is through DI. Moreover, when configuring the dependency injection, the parsed configuration usually gets binded to a strongly typed class by using services.Configure<<strongly typed settings class>>(configuration)
so it's a dog chasing its tail (for decrypting configuration you need DI, for configuring DI, you need the configuration parsed in order to bound it to a strongly typed class). The only solution I came up for now is reconfiguring a second DI IServiceProvider
just for the Data Protection API and use it inside ProtectedConfigurationProvider
. To configure the second DI IServiceProvider
, you have two options:
- You create it by yourself (by instantiating a
ServiceCollection and
calling AddDataProtection
on it) - You let
ProtectedConfigurationProvider
to create it by passing a dataProtectionConfigureAction
parameter. In this case, in order to avoid duplicated code, the configuration of Data Protection API can be performed inside a common private
method called ConfigureDataProtection
, e.g.:
private static void ConfigureDataProtection(IDataProtectionBuilder builder)
{
builder.UseCryptographicAlgorithms(new AuthenticatedEncryptorConfiguration
{
EncryptionAlgorithm = EncryptionAlgorithm.AES_256_CBC,
ValidationAlgorithm = ValidationAlgorithm.HMACSHA256,
}).SetDefaultKeyLifetime(TimeSpan.FromDays(365*15)).PersistKeysToFileSystem
(new DirectoryInfo("..\\..\\Keys"));
}
Here, I chose to use AES 256 symmetric encryption with HMAC SHA256 as digital signature function. Moreover, I ask to store the master encryption key metadata (key, iv, hash algorithm, etc.) in an XML file inside the Keys folder of the console app (by default, keys are stored in a particular location according to MS key management documentation, note that all these APIs are provided by default by the Data Protection API). So when you start the app for the first time, the Data Protection API creates automatically the master encryption key and stores it in the Keys folder, in the following runs, it loads it from this XML file. This configuration however is not the best approach from the security viewpoint because the metadata are stored in plain text, if you want to encrypt the master key at rest, you can use the ProtectKeysWithDpapi
extension method (works only in Windows and in this case, it would be encrypted with Windows DPAPI) or ProtectKeysWithCertificate
to encrypt it with a certificate installed on the computer. Please note that even though you can use different encryption keys in the Keys folder, there is just one master key from which all encryption keys are derived using the purpose string crated from the keyNumber
parameter specified either in the ProtectedConfigurationBuilder
or in the WithProtectedConfigurationOptions
extension method.
In the console application, I add the six configuration sources in the following order to exemplify the merging feature of ASP.NET Core Configuration and also the use of encrypted values:
AddCommandLine
: to add the command line arguments. AddJsonFile
: to add the two json files appsetting.json and appsettings.development.json, the second one has the reloadOnChange
flag set to true
in order to allow the reload of json file whenever it changes on the filesystem.
If you look at the ConnectionStrings section of appsetting.json, there are three keys:
PlainTextConnectionString
: As the name states, it contains a plaintext connection string PartiallyEncryptedConnectionString
: As the name states, it contains a mixture of plain text and multiple Protect:{<data to encrypt>}
tokenization tags. On every run, these tokens get automatically encrypted and replaced with the Protected:{<encrypted data>}
token after the call to the extension method IDataProtect.ProtectFiles.
FullyEncryptedConnectionString
: As the name states, it contains a single Protect:{<data to encrypt>}
token spanning the whole connection string which gets totally encrypted after the first run.
If you look at Nullable section of appsetting.development.json, you can find some interesting keys:
Int, DateTime, Double, Bool
: These keys contain respectively an integer, a datetime, a double and a boolean but they are all stored as a string
using a single Protect:{<data to encrypt>}
tag. Hey wait, how is this possible?
Well, chiefly, all the ConfigurationProviders
convert initially any ConfigurationSource
into a Dictionary<String,String>
in their Load
method (please see the property Data of the framework ConfigurationProvider base abstract class, the Load
method also flattens all the hierarchical path to the key into a string
separated by a colon, so for example Nullable->Int
becomes Nullable:Int
). Only later, this dictionary gets converted and binded to a strongly typed class.
The decryption process of ProtectedConfigurationProvider
happens in the middle, so it's transparent for the user and moreover is available on any simple variable type (DateTime
, bool
, etc.). For now, the full encryption of a whole array is not supported, but you can however encrypt a single element converting the array to an array of string
s (have a look at DoubleArray
key).
AddXmlFile
: to add the XML file appsettings.xml AddInMemoryCollection
: to add an in memory dictionary AddEnvironmentVariables
: to add environment variables
The main code of the Protected.ConsoleTest
console application is:
public static void Main(String[] args)
{
args = new String[] { "--EncryptedCommandLinePassword","Protect:{secretArgPassword!\\*+?|{[()^$.#}", "--PlainTextCommandLinePassword","secretArgPassword!\\*+?|{[()^$.#" };
var servicesDataProtection = new ServiceCollection();
ConfigureDataProtection(servicesDataProtection.AddDataProtection());
var serviceProviderDataProtection = servicesDataProtection.BuildServiceProvider();
var dataProtector = serviceProviderDataProtection.GetRequiredService<IDataProtectionProvider>().CreateProtector(ProtectedConfigurationBuilder.DataProtectionPurpose());
var dataProtectorAdditional = serviceProviderDataProtection.GetRequiredService<IDataProtectionProvider>().CreateProtector(ProtectedConfigurationBuilder.DataProtectionPurpose(2));
var memoryConfiguration = new Dictionary<String, String>
{
["EncryptedInMemorySecretKey"] = "Protect:{InMemory MyKey Value}",
["PlainTextInMemorySecretKey"] = "InMemory MyKey Value",
["TransientFaultHandlingOptions:Enabled"] = bool.FalseString,
["Logging:LogLevel:Default"] = "Protect:{Warning}",
["UserDomain"] = "Protect:{DOMAIN\\USER}",
["EncryptedInMemorySpecialCharacters"] = "Protect:{\\!*+?|{[()^$.#}",
["PlainTextInMemorySpecialCharacters"] = "\\!*+?|{[()^$.#"
};
Environment.SetEnvironmentVariable("EncryptedEnvironmentPassword", "Protect:{SecretEnvPassword\\!*+?|{[()^$.#}");
Environment.SetEnvironmentVariable("PlainTextEnvironmentPassword", "SecretEnvPassword\\!*+?|{[()^$.#");
var encryptedArgs = dataProtector.ProtectConfigurationValue(args);
dataProtectorAdditional.ProtectConfigurationValue(memoryConfiguration);
var encryptedJsonFiles = dataProtector.ProtectFiles(".");
var encryptedXmlFiles = dataProtector.ProtectFiles(".", searchPattern: "*.xml", protectRegexString: "OtherProtect:{(?<protectData>.+?)}", protectedReplaceString: "OtherProtected:{${protectedData}}");
dataProtectorAdditional.ProtectEnvironmentVariables();
Debugger.Break();
var configuration = new ProtectedConfigurationBuilder(dataProtectionServiceProvider: serviceProviderDataProtection)
.AddCommandLine(encryptedArgs)
.AddJsonFile("appsettings.json")
.AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("DOTNETCORE_ENVIRONMENT")}.json", false, true)
.AddXmlFile("appsettings.xml").WithProtectedConfigurationOptions(protectedRegexString: "OtherProtected:{(?<protectedData>.+?)}")
.AddInMemoryCollection(memoryConfiguration).WithProtectedConfigurationOptions(dataProtectionServiceProvider: serviceProviderDataProtection, keyNumber: 2)
.AddEnvironmentVariables().WithProtectedConfigurationOptions(dataProtectionServiceProvider: serviceProviderDataProtection, keyNumber: 2)
.Build();
var services = new ServiceCollection();
services.Configure<AppSettings>(configuration);
var serviceProvider = services.BuildServiceProvider();
var optionsMonitor = serviceProvider.GetRequiredService<IOptionsMonitor<AppSettings>>();
var appSettings = optionsMonitor.CurrentValue;
optionsMonitor.OnChange(appSettingsReloaded => {
Console.WriteLine($"OnChangeEvent: appsettings.{Environment.GetEnvironmentVariable("DOTNETCORE_ENVIRONMENT")}.json has been reloaded! appSettings Int {appSettings.Int} appSettingsReloaded {appSettingsReloaded.Int}");
Debugger.Break();
});
Debugger.Break();
Debug.Assert(appSettings.EncryptedCommandLinePassword==appSettings.PlainTextCommandLinePassword);
Debug.Assert(appSettings.EncryptedEnvironmentPassword == appSettings.PlainTextEnvironmentPassword);
Debug.Assert(appSettings.EncryptedJsonSpecialCharacters == appSettings.PlainTextJsonSpecialCharacters);
Debug.Assert(appSettings.EncryptedXmlSecretKey == appSettings.PlainTextXmlSecretKey);
Debug.Assert(appSettings.EncryptedInMemorySecretKey == appSettings.PlainTextInMemorySecretKey);
int i = 0;
while (i++<5)
{
var environmentAppSettings = File.ReadAllText($"appsettings.{Environment.GetEnvironmentVariable("DOTNETCORE_ENVIRONMENT")}.json");
environmentAppSettings = new Regex("\"Int\":.+?,").Replace(environmentAppSettings, $"\"Int\": \"{dataProtector.ProtectConfigurationValue($"Protect:{{{new Random().Next(0, 1000000)}}}")}\",");
File.WriteAllText($"appsettings.{Environment.GetEnvironmentVariable("DOTNETCORE_ENVIRONMENT")}.json", environmentAppSettings);
Thread.Sleep(5000);
appSettings = optionsMonitor.CurrentValue;
Console.WriteLine($"ConfigurationReloadLoop: appSettings Int {appSettings.Int}");
Debugger.Break();
}
}
The above code is quite simple and commented, if you launch it in Debug mode, it will automatically break into most significant points by using Debugger.Break()
:
- The first one happens after all six configuration sources values have been encrypted by replacing the default tokenization tag
Protect:{<data to encrypt>}
with Protected:{<encrypted data>}
(for files, please check those inside bin/Debug/net6.0 folder and not inside the solution dir which remain unchanged)
- CommandLine arguments
args
gets encrypted in encryptedArgs
variable by using the provided extension method IDataProtect.ProtectConfigurationValue
- The appsettings.*json files have been backed up in a .bak file and encrypted by using the provided extension method
IDataProtect.ProtectFiles
- The appsettings.xml file has been backed up in a .bak file and has been encrypted by using the provided extension method
IDataProtect.ProtectFiles
- The environment variables have got encrypted by using
IDataProtect.ProtectEnvironmentVariables
- The second one is the one actually showing the use of
ProtectedConfigurationBuilder
, if you watch in the debugger the appSettings
strongly typed class, you will notice that it magically and automatically contains the decrypted values with the right data type even though the encrypted keys are always stored in configuration sources as strings
. Please note that I used IOptionsMonitor<AppSettings>
instead IOptions<AppSettings>
because I wanted to test also the autodecryption feature on reload of a configuration file whenever it changes on filesystem. - The third one is inside the
IOptionsMonitor.OnChange
event and happens just after having updated the "Int
" property with a random encrypted integer inside appsettings.development.json file (take note of this value, you will need it again in the next breakpoint), you can see that the appSettingsReloaded
variable contains a different value from the one of appSettings
variable. - The fourth one is after
IOptionsMonitor.OnChange
event and happens after re-assignment to appSettings
variable of the current strongly typed configuration class from IOptionsMonitor
, you can check that appSettings
contains the same new value for "Int
" property as that of appSettingsReloaded
variable at previous breakpoint.
So summing up in order to use this package, we had just to use to replace the call new ConfigurationBuilder()
with a call to new ProtectedConfigurationBuilder()
, pass the Data Protection API configuration and an eventual custom tokenization tag, and everything works flawlessly in a transparent way. Moreover, all the decryption happens in memory and nothing is stored on disk for any reason.
Implementation Details
I explain here the main points of the implementation:
- Extensions methods defined inside
ConfigurationBuilderExtensions
:
IDataProtect.ProtectFiles
is an extension method which gets called and scans all files according to the supplied extension searchPattern
inside the supplied path
for Protect:{<data to encrypt>}
tokens, encrypts enclosed data, performs the replacement with Protected:{<encrypted data>}
and saves back the file after having created an optional backup of the original file with the .bak extension. Again, if you do not like the default tokenization regular expression, you can pass your own one with the constraint that it must extract the <data to encrypt>
substring in a group called protectData
. The format of input files is decoded according to the handlers specified in the public static property ConfigurationBuilderExtensions.FilesDecoding
; by default two "parsers" are provided, one for JSON files (e.g. \\ becomes \, etc., see JsonDecode
function below) and one for XML files (e.g. > becomes >, etc., see XmlDecode
function below). You can add as many decoding handlers as you wish by adding records to the FilesDecoding
list specifying a filenameRegex
which if matched applies the associated dataToEncryptDecodingFunction
, e.g. a function which essentially takes a string as input and returns the parsed string as output. Note that all the handlers are processed in FIFO (First-In First-Out) order so keep this in mind when configuring your additional handlers.
public static (Regex filenameRegex, Func<String, String> dataToEncryptDecodingFunction) JsonDecodingFunction = (new Regex("(.*)\\.json"), JsonDecode);
public static (Regex filenameRegex, Func<String, String> dataToEncryptDecodingFunction) XmlDecodingFunction = (new Regex("(.*)\\.xml"), XmlDecode);
public static List<(Regex filenameRegex, Func<String, String> dataToEncryptDecodingFunction)> FilesDecoding { get; private set; } = new List<(Regex filenameRegex, Func<string, string> dataToEncryptDecodingFunction)>()
{
JsonDecodingFunction,
XmlDecodingFunction
};
public static IList<String> ProtectFiles(this IDataProtector dataProtector, String path, String searchPattern = "*.json", SearchOption searchOption = SearchOption.TopDirectoryOnly, String protectRegexString = null, String protectedReplaceString = "Protected:{${protectedData}}", bool backupOriginalFile = true)
{
var protectRegex = new Regex(protectRegexString ?? ProtectedConfigurationBuilder.DefaultProtectRegexString);
if (!protectRegex.GetGroupNames().Contains("protectData"))
throw new ArgumentException("Regex must contain a group named protectData!", nameof(protectRegexString));
var result = new List<String>();
foreach (var f in Directory.EnumerateFiles(path, searchPattern, searchOption))
{
var fileContent = File.ReadAllText(f);
var extension = Path.GetExtension(f);
var replacedContent = protectRegex.Replace(fileContent, (me) =>
{
var value = me.Groups["protectData"].Value;
foreach (var protectFileDecode in FilesDecoding)
if (protectFileDecode.filenameRegex.Match(f).Success)
{
value = protectFileDecode.dataToEncryptDecodingFunction(value);
break;
}
return protectedReplaceString.Replace("${protectedData}", dataProtector.Protect(value));
});
if (replacedContent != fileContent)
{
if (backupOriginalFile)
File.Copy(f, f + ".bak", true);
File.WriteAllText(f, replacedContent);
result.Add(f);
}
}
return result;
}
public static string XmlDecode(string value)
{
var xmlReader = XmlReader.Create(new StringReader($"<?xml version=\"1.0\"?><root>{value}</root>"));
if (xmlReader.Read())
{
xmlReader.MoveToContent();
value = xmlReader.ReadElementContentAsString();
}
return value;
}
public static string JsonDecode(string value)
{
var jsonReader = new Utf8JsonReader(Encoding.UTF8.GetBytes($"\"{value}\""), new JsonReaderOptions { AllowTrailingCommas = true, CommentHandling = JsonCommentHandling.Skip });
if (jsonReader.Read())
value = jsonReader.GetString();
return value;
}
IDataProtect.ProtectConfigurationValue
is an extension method encrypts a string
, an IEnumerable<string>
, a string[]
or a Dictionary<string,string>
with the same criteria.
public static String ProtectConfigurationValue
(this IDataProtector dataProtector, String value,
String protectRegexString = null,
String protectedReplaceString =
ProtectedConfigurationBuilder.DefaultProtectedReplaceString)
{
var protectRegex = new Regex(protectRegexString ??
ProtectedConfigurationBuilder.DefaultProtectRegexString);
if (!protectRegex.GetGroupNames().Contains("protectData"))
throw new ArgumentException("Regex must contain a group
named protectData!", nameof(protectRegexString));
return protectRegex.Replace(value, (me) => protectedReplaceString.Replace
("${protectedData}", dataProtector.Protect(me.Groups["protectData"].Value)));
}
public static IEnumerable<String> ProtectConfigurationValue
(this IDataProtector dataProtector, IEnumerable<String> arguments,
String protectRegexString = null, String protectedReplaceString =
ProtectedConfigurationBuilder.DefaultProtectedReplaceString)
{
return arguments?.Select
(argument => dataProtector.ProtectConfigurationValue
(argument, protectRegexString, protectedReplaceString));
}
public static String[] ProtectConfigurationValue
(this IDataProtector dataProtector, String[] arguments,
String protectRegexString = null,
String protectedReplaceString =
ProtectedConfigurationBuilder.DefaultProtectedReplaceString)
{
return arguments?.Select(argument =>
dataProtector.ProtectConfigurationValue
(argument, protectRegexString,
protectedReplaceString)).ToArray();
}
public static void ProtectConfigurationValue
(this IDataProtector dataProtector, Dictionary<String, String> initialData,
String protectRegexString = null, String protectedReplaceString =
ProtectedConfigurationBuilder.DefaultProtectedReplaceString)
{
if (initialData != null)
foreach (var key in initialData.Keys.ToList())
initialData[key] = dataProtector.ProtectConfigurationValue
(initialData[key], protectRegexString, protectedReplaceString);
}
IDataProtect.ProtectEnvironmentVariables
is an extension method which encrypts the environment variables with the same criteria.
public static void ProtectEnvironmentVariables
(this IDataProtector dataProtector, String protectRegexString = null,
String protectedReplaceString =
ProtectedConfigurationBuilder.DefaultProtectedReplaceString)
{
var environmentVariables = Environment.GetEnvironmentVariables();
if (environmentVariables != null)
foreach (string key in environmentVariables.Keys)
Environment.SetEnvironmentVariable
(key, dataProtector.ProtectConfigurationValue
(environmentVariables[key].ToString(), protectRegexString,
protectedReplaceString));
}
IConfigurationBuilder.WithProtectedConfigurationOptions
is an extension method which allows to override the Data Protection or tokenization tag configuration for a particular ConfigurationSource
(e.g., the last one added). Note that this method is a little bit hacky: I was not able to change the return type of ProtectedConfigurationBuilder.Add
, otherwise the interface IConfigurationBuilder
wouldn't be implemented; thus WithProtectedConfigurationOptions
extend the standard IConfigurationBuilder
interface and converts it to IProtectedConfigurationBuilder
interface and calls the WithProtectedConfigurationOptions
, if the provided IConfigurationBuilder
is not an instance of IProtectedConfiguration
, it raises an exception remembering to replace the new ConfigurationBuilder
instantiation with new ProtectedConfigurationBuilder
.
public static IConfigurationBuilder WithProtectedConfigurationOptions
(this IConfigurationBuilder configurationBuilder,
String protectedRegexString = null,
IServiceProvider dataProtectionServiceProvider = null,
Action<IDataProtectionBuilder> dataProtectionConfigureAction = null,
int keyNumber=1)
{
var protectedConfigurationBuilder = configurationBuilder as
IProtectedConfigurationBuilder;
if (protectedConfigurationBuilder != null)
return protectedConfigurationBuilder.WithProtectedConfigurationOptions
(protectedRegexString, dataProtectionServiceProvider,
dataProtectionConfigureAction, keyNumber);
else
throw new ArgumentException("Please use ProtectedConfigurationBuilder
instead of ConfigurationBuilder class!", nameof(configurationBuilder));
}
ProtectedConfigurationBuilder
implements the IConfigurationBuilder
interface like ConfigurationBuilder
framework class (from which part of the implementation is borrowed), the main difference is the Build
method which elementally proxies through composition the IConfigurationProvider
returned from the IConfigurationSource.Build
method by passing it as a constructor parameter to the core class responsible for in memory transparent decryption: ProtectedConfigurationProvider
. It also performs the merge between the custom configuration specified for the IConfigurationSource
being converted into IConfigurationProvider
(if you want to know how, check the ProtectedConfigurationData.Merge
static
method) and the global configuration specified in the ProtectedConfigurationBuilder
constructor.
public virtual IConfigurationRoot Build()
{
var providers = new List<IConfigurationProvider>();
foreach (IConfigurationSource source in _sources)
{
IConfigurationProvider provider = source.Build(this);
ProtectedProviderConfigurationData.TryGetValue
(source.GetHashCode(), out var protectedConfigurationData);
if (protectedConfigurationData != null)
{
ProtectedProviderConfigurationData[provider.GetHashCode()] =
protectedConfigurationData;
ProtectedProviderConfigurationData.Remove(source.GetHashCode());
}
providers.Add(CreateProtectedConfigurationProvider(provider));
}
return new ConfigurationRoot(providers);
}
protected IConfigurationProvider CreateProtectedConfigurationProvider
(IConfigurationProvider provider)
{
var actualProtectedConfigurationData =
ProtectedProviderConfigurationData.ContainsKey(provider.GetHashCode()) ?
ProtectedConfigurationData.Merge(ProtectedGlobalConfigurationData,
ProtectedProviderConfigurationData[provider.GetHashCode()]) :
ProtectedGlobalConfigurationData;
if (actualProtectedConfigurationData?.IsValid != true)
return provider;
return new ProtectedConfigurationProvider
(provider, actualProtectedConfigurationData);
}
ProtectedConfigurationData
: This class chiefly stores either the global configuration or the ConfigurationSource
specific one for Data Protection API and the eventual custom tokenization tag. It also sets up another dependency injection provider in the constructor (see above for the reason):
public ProtectedConfigurationData(String protectedRegexString = null,
IServiceProvider dataProtectionServiceProvider = null,
Action<IDataProtectionBuilder>
dataProtectionConfigureAction = null,
int keyNumber=1)
{
if (String.IsNullOrEmpty(protectedRegexString) &&
dataProtectionServiceProvider == null && dataProtectionConfigureAction == null)
throw new ArgumentException("Either protectedRegexString or
dataProtectionServiceProvider
or dataProtectionConfigureAction must not be null!");
if (dataProtectionServiceProvider == null &&
dataProtectionConfigureAction != null)
{
var services = new ServiceCollection();
dataProtectionConfigureAction(services.AddDataProtection());
dataProtectionServiceProvider = services.BuildServiceProvider();
}
if ((dataProtectionServiceProvider != null) &&
((DataProtector = dataProtectionServiceProvider.GetRequiredService
<IDataProtectionProvider>().CreateProtector
(ProtectedConfigurationBuilder.DataProtectionPurpose(keyNumber))) == null))
throw new ArgumentException
("Either dataProtectionServiceProvider or dataProtectionConfigureAction must
configure the DataProtection services!",
dataProtectionServiceProvider == null ?
nameof(dataProtectionServiceProvider) : nameof(dataProtectionConfigureAction));
ProtectedRegex = new Regex(protectedRegexString ??
ProtectedConfigurationBuilder.DefaultProtectedRegexString);
if (!ProtectedRegex.GetGroupNames().Contains("protectedData"))
throw new ArgumentException("Regex must contain a group
named protectedData!",
nameof(protectedRegexString));
}
ProtectedConfigurationProvider
: This class performs the actual transparent decryption of encrypted configuration values stored inside any existing configuration source. It should also support even future configuration sources as long as the implementation of the GetChildKeys of Microsoft.Extensions.Configuration.ConfigurationProvider does not change (I principally use it to enumerate all possible configuration keys). Essentially, it takes in its constructor as input the IConfigurationProvider
that needs to be decrypted and acts as a proxy:
-
It redefines the Load
method by calling after input IConfigurationProvider.Load
the method responsible for the actual decryption DecryptChildKeys
. This method uses the standard method GetChildKeys
offered by the IConfigurationProvider
interface to enumerate all the existing configuration keys and decrypt them by using the protectedRegex
stored inside ProtectConfigurationData
and IDataProtect.Unprotect
method.
public void Load()
{
Provider.Load();
DecryptChildKeys();
}
protected void DecryptChildKeys(String parentPath = null)
{
foreach (var key in Provider.GetChildKeys(new List<String>(),
parentPath).Distinct())
{
var fullKey = parentPath != null ? $"{parentPath}:{key}" : key;
if (Provider.TryGet(fullKey, out var value))
{
if (!String.IsNullOrEmpty(value))
Provider.Set(fullKey,
ProtectedConfigurationData.ProtectedRegex.Replace
(value, me =>
ProtectedConfigurationData.DataProtector.Unprotect
(me.Groups["protectedData"].Value)));
}
else DecryptChildKeys(fullKey);
}
}
-
It creates its own ReloadToken
if the underlying IConfigurationProvider
supports it, it returns it in the GetReloadToken
method and finally it registers a callback to the input IConfigurationProvider
reload token using the framework static utility method ChangeToken.OnChange
class in order to be notified of any configuration change re-performing the decryption in order to support to automatic decryption of values on configuration reload
public ProtectedConfigurationProvider(IConfigurationProvider provider, ProtectedConfigurationData protectedConfigurationData)
{
Provider = provider;
ProtectedConfigurationData = protectedConfigurationData;
RegisterReloadCallback();
}
protected void RegisterReloadCallback()
{
if (Provider.GetReloadToken() != null)
{
ReloadToken = new ConfigurationReloadToken();
ProviderReloadTokenRegistration = ChangeToken.OnChange(() => Provider.GetReloadToken(), (configurationProvider) =>
{
var protectedConfigurationProvider = configurationProvider as ProtectedConfigurationProvider;
protectedConfigurationProvider.DecryptChildKeys();
OnReload();
}, this);
}
}
public IChangeToken GetReloadToken()
{
return ReloadToken;
}
protected void OnReload()
{
ConfigurationReloadToken previousToken =
Interlocked.Exchange(ref ReloadToken, new ConfigurationReloadToken());
previousToken.OnReload();
}
Points of Interest
I think that the idea of specifying the custom tag through a regex is very witty because it gives every user the flexibility they need to customize the tokenization tag. I have also released it as a NuGet package on NuGet.Org.
It's strange that MS did not plan a method or a property inside IConfigurationProvider
to enumerate all possible keys of a provider,
probably (and hopefully), it will be added in future releases so I could avoid using the non efficient recursive GetChildKeys
method to enumerate all keys.
If you wonder whether you can use this package in your project and it will still be working in the future, I can underline that the only critical point is the enumeration of all configuration keys which is now done by using the GetChildKeys
method. Even if its implementation could be changed by Microsoft, it will always provide what its name states, e.g., the child keys of a configuration key. And even in the remote case that the GetChildKeys
method will be removed from the IConfigurationProvider
interface, you can always access all configuration keys by casting the interface to the ConfigurationProvider
base class (please see the comment in CreateProtectedConfigurationProvider
method, basically all configuration providers derive from this class), so I am quite confident that the package will be working for many years.
Last but not the least, I want to remind everyone that the protected
member access modifier does exist! I have seen it being used rarely as compared to private
, sometimes even inside the .NET framework classes! It should be used instead as the default member access modifier with very little or almost none private
members, since it allows inheritance
which is the basis of object-oriented programming (you never know who wants to extend and customize your classes for their or other general needs).
History
- V1.0.0 (16th December, 2023)
- V1.0.1 (27th December, 2023)
- Added further code in implementation details sections of
ConfigurationBuilderExtensions, ProtectedConfigurationBuilder
and ProtectedConfigurationProvider
- Explained better how automatic decryption of values on configuration reload works in implementation details sections
- V1.0.2 (30th December, 2023)
- Improved the
ConfigureDataProtection
explanation and added link to Data Protection purpose string documentation - Added point in "Points of Interest" section about missing method or property on MS
IConfigurationProvider
to enumerate all keys - Fixed some typos
- V1.0.3 (7th February, 2024)
- Commented 3 lines of unneeded code in
CreateProtectedConfigurationProvider
method of ProtectedConfigurationBuilder
- Released NuGet package version 1.0.5
- V1.0.4 (8th February, 2024)
- Added point in Points of Interest section about the critical points of this package and its likelihood to be still working in the future
- Added point in Points of Interest section about fostering the usage of protected member access modifier
- V1.0.5 (4th May, 2024)
- Released NuGet package version 1.0.6
- Updated paragraphs and code to reflect code changes (e.g.
ConfigurationBuilderExtensions
. IDataProtect.ProtectFiles
and Protected.ConsoleTest
) - Added source code hyperlinks to appsetting.json, appsettings.development.json and appsettings.xml files
- V1.0.6 (8th May, 2024)
- Released NuGet package version 1.0.7
- Updated paragraphs and code to reflect code changes (e.g.
ConfigurationBuilderExtensions
, ProtectedConfigurationProvider.RegisterReloadCallback
and Protected.ConsoleTest
)