The Server

The configuration system uses an HTTP server to provide section information to clients. The server can read the data from files on disk, or the data can theoretically come from virtually anywhere via custom Section Providers.

In addition to the section name requested, the server can also use "environments" to return different configurations to different clients. This allows for the same config servers to provide configuration for staging and production, as well as being able to have different groups of servers running with different configuration versions.

Finally, the servers also have the capability to slowly roll out a new configuration value. There are cases where you do not want a value to be changed all at once for all clients, and this was historically accomplished by simply pushing the new file out with a delay between groups of servers. For the configuration servers, you can provide at time by which all clients should have the new config. As requests come in, clients are randomly assigned into the "new" group with a probability based on how close the current time is to the completed time.

All of this functionality is controlled with the "section mapping", an xml config file that the servers can either read from the local file system or any other source that a provider exists for.

The basic logic for serving a request for a give section is:
  1. Check to see which environment the requester's address is in. If not found, use the default environment.
  2. Get the requested configuration section's information.
  3. If there is an "exception" defined for the environment the client is in, use that exception's source value. Otherwise use the default.
  4. If the client supplied a hash with the request and it matches the hash of the configuration that would returned, simply return an HTTP 304 ("Not Changed") value
  5. Otherwise, return the configuration information that was found.

Basic Server Configuration

By default the server will run, listening on port 9000 and using SectionMapping.xml for the section definitions and a few other sensible defaults. If you'd like to change any of these, you will need to add the ConfigurationSystemServerConfig to your application configuration.

The section definition is

<section name="ConfigurationSystemServerConfig" type="MySpace.ConfigurationSystem.ConfigurationSystemServerConfig, MySpace.ConfigurationSystem.Server"/>

All of the attributes on the configuration itself are optional; here is an example with all of the defaults. Information about each attribute is below.

<ConfigurationSystemServerConfig    
  port="9000"
  prefix="*"
  compressionThreshold="10240"  
  mappingConfigPath="SectionMapping.xml"    
  mappingConfigProvider="FileProvider"
  preCaching="true"
  keyProviderTypeName="MySpace.ConfigurationSystem.Encryption.Keys.KeyProvider, MySpace.ConfigurationSystem.Encryption.Keys"
 />
  • port
    • The port the server will listen on.
  • prefix
  • compressionThreshold
    • The server will gzip any response that it larger than the compressionThreshold in bytes for clients that indicate they support it via the standard Accept-Encoding="gzip" header.
  • mappingConfigPath
    • The source value that will be passed to the mappingConfigProvider
  • mappingConfigProvider
    • The name of the provider used to fetch the section mapping. Please see Section Providers for more information.
    • precaching
  • By default the server will fetch a copy of each config for the default environment on start up. If you wish to disable this (if, for example, you have a large number of rarely used sections) set this to false.
  • keyProviderTypeName
    • The name of the key provider used for encryption. Please see Section Encryption for more information.

If you look a the schema and the configuration definition, you will notice that there are a handful of additional attributes that start with tfs. These are there to support our internal TFS provider which is not currently released. A method to support generic provider configuration and decouple this from the main config would be a excellent update to the server but we have not gotten there yet!

Section Mapping

The section mapping contains the details configuration that defines what and how configuration information is accessed by the server and sent to the clients. The section mapping is read by the server using the same mechanism as the configuration sections themselves - a path is sent to an implementation of IConfigurationSectionProvider which returns the data. By default the included FileProvider is used to access SectionMapping.xml. Please see Section Providers for more information on this.

Because the section provider might be reading data remotely - for example we keep the our Section Mapping in TFS - the server keeps a local copy as backup. On shut down its written to the file MappingXml.backup.[AssemblyVersion].xml, and on start up if the provider cannot provide the section mapping the backup will be used. The file is versioned so that if the schema changes in a subsequent code update, simply changing the server assembly's version number will prevent it from being used. The provider is checked every minute to see if the mapping has been updated, and if it has changed it will be reloaded.

The mapping file has two sections - the environment definition and the section mappings themselves. In its most basic form, the xml looks like:

<?xml version="1.0" encoding="utf-8" ?>
<configurationsystem>
  <environments>
     <environment name="EnvironmentName" default="true" comment="default environment"/>
  </environments>
  <sectionmapping>
    <section
      name="SectionName"
      handler="ProviderName"
      source="Info for the provider to get the information"
      comment="A description of this section"
    />
  </sectionmapping>
</configurationsystem>

Environments

The environments define groups of servers, and are used to serve different information for the same section name when necessary. When a request for a given section is received, it it first checked against the defined environments. If the requester's endpoint is found in one of them, then that one is used. Otherwise, it is assumed to be in the environment with "default" set to true. It is the sections that define what information goes to which environment, see below for detail on that.

Since netscalers are used heavily in our environment, we check the "Client-IP" header before using the actual endpoint that made the request. A netscaler will set this value to be the actually originator.

Environments consist of a list of single endpoints and Active Directory groups. For a small environment, simply listing each server by address probably simplest. Such an environment looks like

<environment name="Release" default="false">
  <endpoint address="releaseserver1.domain"/>
  <endpoint address="releaseserver2.myspace.com"/>
</environment>

You should create one environment with the "default" attribute set to "true". This is the environment that clients will be assumed to be in if they are not found in any others.

The types of definitions that the environment node supports are:
  • Endpoint
    • A simple server address, which can be a DNS name, IPv4 or IPv6 address.
  • VIP
    • This is to support MySpace's internal environment and relies on an internal web service. This should be made configurable in order to support any web service that returns a list of server endpoints!
  • DirectoryGroup
    • This is for querying active directory to get a list of servers. You must supply a root and filter via attributes of the same name to be passed to a System.DirectoryServices.DirectorySearcher object. Please see Microsoft's documentation for more information.

Sections

The SectionMapping node defines the actual configuration sections served by the configuration system. In order to provide as much flexibility as possible, sections are read by the server using Section Providers, which are just implementations of a simple interface. A section provider for reading off of local files is provided as a starting point and example.

A section definition looks like this:

<section
name="MySection"
handler="FileProvider"
source="MySection.config"
comment="For the XmlSerializerExample project"
generic="false"
encrypt="false"
<exception environment="Development" source="MySection.Development.config" />
/>
  • name
    • This is the name of the section and the key used when accessing it, e.g. http://configserver/get/MySection
  • provider
    • This is the name of the class that the server uses to read the section. "FileProvider" is built in; please see Section Providers for creating others.
  • source
    • This is the information provided to the provider in order to read the section. In the case of FileProvider it is a simple path relative to the server root.
  • comment
    • Nothing more than information about the section, to make administering them easier. This is optional.
  • generic
    • The configuration client understands sections that are presented in the key/value appSettings format. Please see Generic Sections for more information.
  • encrypt
    • The configuration server can serve sections encrypted to provide some measure of protection for sensitive information. Please see Section Encryption for more information.

Exceptions

Exceptions in section definitions provide two pieces of functionality: providing configuration for clients in a particular environment from a different source, and allows for gradual deployment of new configs.

If you want some clients, say a testing environment, to be supplied with different configuration values, simply provide the name of environment (as defined above) and the alternate source value as attributes to an "exception" node.

Slow Rolls

In some circumstances you don't want an entire farm to receive a new configuration at once. For example, if you are redirecting traffic from one middle tier system to another, you may wish to allow for the new system to warm up instead of being slammed with new traffic. Section definitions support a special kind of exception, called a "slowroll" to accomplish this.

A slow roll is defined with a souce value of the new configuration, a start and end time, and an optional environment name. During the window between the start and end time, as clients request new configs they are gradually and randomly assigned to the "new" bucket. For example, if 10% of the window has elapsed, there is a 10% chance that a given client which has not already gotten the new config will. This is by no means perfect, exact, or even linear, but it's a simple way to gradually push a new config out.
  • The name of the node under the exceptions node should be "slowroll" and contain the following attributes
    • environment
      • Optional. The environment participating in the slow roll. If this is not supplied the default environment will be used.
    • source
      • The new source to be supplied to the provider
    • start
    • end
      • The start and end times of the slow roll. These will be parsed with DateTime.TryParse. If parsing fails or they are not supplied, DateTime.Now will be used.

Other

You may notice that the XSD for the section mapping contains three additional elements: template, xsdList, and exception. These are to support our currently unreleased section management system. You can safely ignore them, because they are not used directly by the server.

Last edited May 24, 2011 at 10:17 PM by eriknelson, version 10

Comments

No comments yet.