Unknown Class Dynamically Generated at runtime for Entity Framework, WCF OData and dynamic client consumption

• Download DynamicClassGenerator.zip - 103.2 KB

Introduction

I had the pleasure on working on one of the most complex things I or most developers would not want to work on.  I have worked on projects before that had some similarities, but not to the extent that this requirement went to.

A few things to keep in mind through this article is that I will not supply the actual constructs of the various tests and performance calculations, but rather the end conclusion of how to dynamically generate classes, store data, expose it via WCF and consume it client side.  This is a very slim-lined solution and will be extended, since I am still busy working on it, and is only to give some insight to the thought process.

Background

The Brief/Requirement:

The application might or might not receive a file containing some construct.  If there is no file, the user must be able to configure it up front.  The data must be stored for comparison and viewable.  It has to be fast and handle large volumes of data.  The data must be REST (JSON) accessible and you must be able to query the data through a browser.

The concerns:

1. Some data
2. REST
3. Must be fast
4. Large amounts of data

The components that will be affected:

1. Dynamic data store
2. Dynamic web service
3. Dynamic data handling on client side

Okay, so basically I need to grab some “air” and do something with it.  Now that should not be too hard (some sarcasm intended).

Investigation:

1. Database

The first and immediate approach was to have a look again at EAV (Entity Attribute Value) design for the database.  I know most developers (database admins) will take a gun and point it to my head, since this should be the last resort, I know.  But for the interest sake of it, you can store the meta information in tables and then store the data row based. 

We took a file with about 80,000 records and 35 fields as a test base and ported the data to a normal “wide” table and an EAV type table and proceeded to replicate the data 4 times.  Resulting in just over 250,000 records in the “wide” table and over 7,900,000 in the EAV value table after omitting empty values.

Damn, was I surprised at how slow it was retrieving that amount of data out of the EAV construct, no wonder database administrators are not fond of the idea.

Okay, there goes EAV out of the window and here comes traditional “wide” tables.

Now, remember that we will never now the table structure being received!!!!

2. Class/Object use:

The next thing was on how I use the data, and how to populate it for code use.  Now I did some reading up on Expando, not much to be honest, since it looked to me, very similar of having an EAV structure with a Dictionary<string, object> for each object.  Now if it was bad for the database, would it not be as bad when done in code.  With the limited timeframe, I did not even want to go down that route of testing, since I did stumble on a nice article on how to use the ILGenerator to create dynamic classes at runtime. (Code included)

The benefit is that you can build a class and use ILSpy to look at a class and then build a construct to build the equivalent code at runtime, and compare the result.

My conclusion was, to generate the class and properties from meta information stored and add the relevant attributes so it is database and WCF aware and for good measure just add a base class that implement the INotifyPropertyChanged interface with an ID (key) property.

Another big benefit is that if you put the ILGenerator code in a separate assembly it could be used to generate the class on the “fly” both server and client side.

The next BIG question is, what did I as a developer want to do?

Well there is actually a few, and it is all the goodies that are out there that we would normally use in most of our application development.

1. I want to use Entity Framework as much as possible, why still write your own ORM if Microsoft so neatly gave us one to work with.
2. I want to use WCF Data Services for Entity Framework, since it neatly couples together and since I don’t have to write various OperationContract’s to be exposed, I can just expose the data with Data Services.

On top of it, I will not be able to do so, since the data tables and classes will be done at runtime.

3. I want to use Linq to query the data services or get very close to it.

Now the question is why I want to do something as stupid as that.  Well, the answer is simple, every developer knows some or all of the constructs that I mention, so in case I get hit by a bus, someone else must be able to continue without having to learn some crappy construct.

All of this said, at least we can lay some ground rules (now these rules can change over time as the system develops and constraints are put in place, but not in this example):

1. You cannot make anything out of air, so some information will need to be supplied.
2. We will need 2 constructs, one for the meta information and one for the “dynamic” content.
3. In the beginning no relational constructs will be dealt with.
4. We will not allow the table to be modified once data has been added

Now then, with a certain picture in mind, and I do hope you have it too, it is putting my head down and slowly putting all of this together.

The proposed solution

Firstly, the meta information.

We are going to use some of the EAV structure, but only to store information about our table that will be generated and used.  We need a Template table which in this case is the destination table name, an Attribute table that will tell us what the field names are and a TemplateAttributes table that links the Template and Attribute table.  This table is added, since a field may or may not belong to multiple tables with different field attributes.  There is an additional Type lookup table that we can use later on to make .Net and SQL types at a later stage.

In this example I was basing the information from a csv file, so the Idx column is for the index in the file, for easier use later on.  The DisplayName will be used to show a different name on the grid later on, by building the grid up with a helper class, none of this will be used for this example.

The rest of the fields in the tables should be used.

At least we have something to start working with.

Building the runtime class

Now we are ready to start using the ILGenerator to build our class up.

As mentioned, we will be using a base class to implement INotifyPropertyChanged and expose an Id property.  In this case I made an abstracted class for the base class in case I need it later, and force certain implementations in future to adhere to some constraints.

public abstract class DynamicEntity : INotifyPropertyChanged
{
    public abstract int Id { get; set; }
    public abstract event PropertyChangedEventHandler PropertyChanged;
    public abstract void OnPropertyChanged(string propertyName);
}

public class BaseDynamicEntity : DynamicEntity
{
    private int _id;

    [DataMember, Key]
    public override int Id
    {
        get
        {
            return _id;
        }
        set
        {
            _id = value;
            OnPropertyChanged("Id");
        }
    }

    public override event PropertyChangedEventHandler PropertyChanged;

    public override void OnPropertyChanged(string propertyName)
    {
        if (PropertyChanged != null)
        {
            PropertyChanged.Invoke(this, new PropertyChangedEventArgs(propertyName));
        }
    }
}

Now we can start building the class, the use of the various builders are not being dealt with, since they are included in the references.

VERY IMPORTANT
For assembly references and the WCF Data Service, it is vital that the runtime generated classes reference the same assemblies, if they don’t you will get errors

I use the WCF Data Services supplied by Microsoft from Nuget, for this project you need to install and use:

There is various overloads and non-generic methods in the main class, but for the article the following will work.  The methods are public with return types, so you can extend, write your own and run each separately and see what it looks like.

public class DynamicClassFactory
{
    private AppDomain _appDomain;
    private AssemblyBuilder _assemblyBuilder;
    private ModuleBuilder _moduleBuilder;
    private TypeBuilder _typeBuilder;
    private string _assemblyName;

    public DynamicClassFactory() : this("Dynamic.Objects")
    {
    }

    public DynamicClassFactory(string assemblyName)
    {
        _appDomain = Thread.GetDomain();
        _assemblyName = assemblyName;
    }

    /// <summary>
    /// This is the normal entry point and just return the Type generated at runtime
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="name"></param>
    /// <param name="properties"></param>
    /// <returns></returns>
    public Type CreateDynamicType<T>(string name, Dictionary<string, Type> properties) where T : DynamicEntity
    {
        var tb = CreateDynamicTypeBuilder<T>(name, properties);
        return tb.CreateType();
    }

    /// <summary>
    /// Exposes a TypeBuilder that can be returned and created outside of the class
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="name"></param>
    /// <param name="properties"></param>
    /// <returns></returns>
    public TypeBuilder CreateDynamicTypeBuilder<T>(string name, Dictionary<string, Type> properties)
        where T : DynamicEntity
    {
        if (_assemblyBuilder == null)
            _assemblyBuilder = _appDomain.DefineDynamicAssembly(new AssemblyName(_assemblyName),
                AssemblyBuilderAccess.RunAndSave);
        //vital to ensure the namespace of the assembly is the same as the module name, else IL inspectors will fail
        if (_moduleBuilder == null)
            _moduleBuilder = _assemblyBuilder.DefineDynamicModule(_assemblyName + ".dll");

        //typeof(T) is for the base class, can be omitted if not needed
        _typeBuilder = _moduleBuilder.DefineType(_assemblyName + "." + name, TypeAttributes.Public
                                                        | TypeAttributes.Class
                                                        | TypeAttributes.AutoClass
                                                        | TypeAttributes.AnsiClass
                                                        | TypeAttributes.Serializable
                                                        | TypeAttributes.BeforeFieldInit, typeof(T));

        //various class based attributes for WCF and EF
        AddDataContractAttribute();
        AddTableAttribute(name);
        AddDataServiceKeyAttribute();

        //if there is a property on the base class and also in the dictionary, remove them from the dictionary
        var pis = typeof(T).GetProperties();
        foreach (var pi in pis)
        {
            properties.Remove(pi.Name);
        }

        //get the OnPropertyChanged method from the base class
        var propertyChangedMethod = typeof(T).GetMethod("OnPropertyChanged", BindingFlags.Instance | BindingFlags.NonPublic);

        CreateProperties(_typeBuilder, properties, propertyChangedMethod);

        return _typeBuilder;
    }

    public void AddDataContractAttribute()
    {
        Type attrType = typeof(DataContractAttribute);
        _typeBuilder.SetCustomAttribute(new CustomAttributeBuilder(attrType.GetConstructor(Type.EmptyTypes),
            new object[] { }));
    }

    public void AddTableAttribute(string name)
    {
        Type attrType = typeof(TableAttribute);
        _typeBuilder.SetCustomAttribute(new CustomAttributeBuilder(attrType.GetConstructor(new[] { typeof(string) }),
            new object[] { name }));
    }

    public void AddDataServiceKeyAttribute()
    {
        Type attrType = typeof(DataServiceKeyAttribute);
        _typeBuilder.SetCustomAttribute(new CustomAttributeBuilder(attrType.GetConstructor(new[] { typeof(string) }),
            new object[] { "Id" }));
    }

    public void CreateProperties(TypeBuilder typeBuilder, Dictionary<string, Type> properties, MethodInfo raisePropertyChanged)
    {
        properties.ToList().ForEach(p => CreateFieldForType(p.Value, p.Key, raisePropertyChanged));
    }

    private void CreateFieldForType(Type type, String name, MethodInfo raisePropertyChanged)
    {
        FieldBuilder fieldBuilder = _typeBuilder.DefineField("_" + name.ToLowerInvariant(), type, FieldAttributes.Private);

        PropertyBuilder propertyBuilder = _typeBuilder.DefineProperty(name, PropertyAttributes.HasDefault, type, null);

        //add the various WCF and EF attributes to the property
        AddDataMemberAttribute(propertyBuilder);
        AddColumnAttribute(propertyBuilder);

        MethodAttributes getterAndSetterAttributes = MethodAttributes.Public | MethodAttributes.SpecialName | MethodAttributes.HideBySig;// | MethodAttributes.Virtual;

        //creates the Get Method for the property
        propertyBuilder.SetGetMethod(CreateGetMethod(getterAndSetterAttributes, name, type, fieldBuilder));
        //creates the Set Method for the property and also adds the invocation of the property change
        propertyBuilder.SetSetMethod(CreateSetMethod(getterAndSetterAttributes, name, type, fieldBuilder, raisePropertyChanged));
    }

    private void AddDataMemberAttribute(PropertyBuilder propertyBuilder)
    {
        Type attrType = typeof(DataMemberAttribute);
        var attr = new CustomAttributeBuilder(attrType.GetConstructor(Type.EmptyTypes), new object[] { });
        propertyBuilder.SetCustomAttribute(attr);
    }

    private void AddColumnAttribute(PropertyBuilder propertyBuilder)
    {
        Type attrType = typeof(ColumnAttribute);
        var attr = new CustomAttributeBuilder(attrType.GetConstructor(Type.EmptyTypes), new object[] { });
        propertyBuilder.SetCustomAttribute(attr);
    }

    private MethodBuilder CreateGetMethod(MethodAttributes attr, string name, Type type, FieldBuilder fieldBuilder)
    {
        var getMethodBuilder = _typeBuilder.DefineMethod("get_" + name, attr, type, Type.EmptyTypes);

        var getMethodILGenerator = getMethodBuilder.GetILGenerator();
        getMethodILGenerator.Emit(OpCodes.Ldarg_0);
        getMethodILGenerator.Emit(OpCodes.Ldfld, fieldBuilder);
        getMethodILGenerator.Emit(OpCodes.Ret);

        return getMethodBuilder;
    }

    private MethodBuilder CreateSetMethod(MethodAttributes attr, string name, Type type, FieldBuilder fieldBuilder, MethodInfo raisePropertyChanged)
    {
        var setMethodBuilder = _typeBuilder.DefineMethod("set_" + name, attr, null, new Type[] { type });

        var setMethodILGenerator = setMethodBuilder.GetILGenerator();
        setMethodILGenerator.Emit(OpCodes.Ldarg_0);
        setMethodILGenerator.Emit(OpCodes.Ldarg_1);
        setMethodILGenerator.Emit(OpCodes.Stfld, fieldBuilder);

        if (raisePropertyChanged != null)
        {
            setMethodILGenerator.Emit(OpCodes.Ldarg_0);
            setMethodILGenerator.Emit(OpCodes.Ldstr, name);
            setMethodILGenerator.EmitCall(OpCodes.Call, raisePropertyChanged, null);
        }

        setMethodILGenerator.Emit(OpCodes.Ret);

        return setMethodBuilder;
    }

    public void SaveAssembly()
    {
        _assemblyBuilder.Save(_assemblyBuilder.GetName().Name + ".dll");
    }
}

Now we have completed the meta information and the ability to create the class at runtime, for both use client and server side.

Entity Framework

Next we need to read the meta information and add a second model for the Dynamic tables, the reason for this is that the meta information can be done with a Model first approach and with EF 6.0, we include all the tables:

This will also automatically add EntityFramework 6.1.1 from NuGet for us.

Next we need to add the database context for reading the future dynamic tables, for this we can’t use the wizard and we can’t use DbSet<T>, since we will never know the classes up front. 

Now what does this mean and I hate to break it to you, we lose the ChangeTracker feature from EF and till there is a way to add it, this will not be there.  This is not the end of the world yet!!!!

Another issue is that we have no schema for mapping the classes to the database. Thanks to a magic assembly written by someone only known as maxbeaudoin, we are able to generate the model “magically”.  So grab the MagicDbModelBuilder, add it as a reference and add it under using.

Create a class that derives from DbContext, add a method to add the types to the context from outside and override OnModelCreating.  In these early stages, the “Id” is hardcoded, but you can look up the DataServiceKeyAttribute and dynamically check for the HasKey option.  Since some constraints was set by the abstract class, I am safe in hard coding the “Id” key check.

public partial class DynamicDbContext : DbContext
{
    public DynamicDbContext()
        : base("name=DynamicDbContext")
    {
        Database.SetInitializer(new NullDatabaseInitializer<DynamicDbContext>());
    }

    public void AddTable(Type type)
    {
        _tables.Add(type.Name, type);
    }

    private Dictionary<string, Type> _tables = new Dictionary<string, Type>();

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        base.OnModelCreating(modelBuilder);
        var entityMethod = modelBuilder.GetType().GetMethod("Entity");

        foreach (var table in _tables)
        {
            entityMethod.MakeGenericMethod(table.Value).Invoke(modelBuilder, new object[] { });
            foreach (var pi in (table.Value).GetProperties())
            {
                if (pi.Name == "Id")
                    modelBuilder.Entity(table.Value).HasKey(typeof(int), "Id");
                else
                    modelBuilder.Entity(table.Value).StringProperty(pi.Name);
            }
        }
    }
}

VERY IMPORTANT
Since we have 2 models and they are differently constructed, there are 2 connection strings, one contains the ssdl information and the other is a plain traditional database connection string with no information about the schema.

WCF Data Service (Server Side)

Unlike most developers, I do not attach my code file to my svc file, since the code file can be made portable to be used for asmx services as well, and there is many articles about it.

Now we will need 2 class files, again one to expose the meta information and another for the dynamic tables.  For this sample, I will be including it in the DynamicServerSide assembly, although this could be anywhere else.

Since I have not sat down to figure out the changes in DataService<T>, I am still using the prerelease of EntityFrameworkDataService<T>.  This can be downloaded with NuGet, just make sure the set to “Include PreRelease” and look for “WCF Data Services EntityFramework Provider”.

Additionally, I am interested in it being working, so security is omitted.

First we have the MetadataService:

[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public partial class DynamicMetadataService : EntityFrameworkDataService<DynamicEntities>
{
    public static void InitializeService(DataServiceConfiguration config)
    {
        config.SetEntitySetAccessRule("*", EntitySetRights.AllRead);
        config.DataServiceBehavior.AcceptProjectionRequests = true;
        config.UseVerboseErrors = true;
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3;
    }
}

Second, the DynamicODataService

I have specifically made it OData, since the ?$format=json, will automatically expose the REST-JSON format as per the brief (whoop).

I have also included a simple CreateTable WebInvoke POST method, so that after saving the meta data into the main database, you can tell the database to actually create the table, nothing fancy.  One thing to note is that since there are 2 EF models, you need to create a database connection to read the meta and create the table.  This is not really a downfall, this is because you can actually create another database to just hold all these “dynamic” tables and keep it away from your main database.

Another vital part to note is the override on CreateDataSource(), this allows us to actually read the meta information and create the classes at runtime.

[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class DynamicODataService : EntityFrameworkDataService<DynamicDbContext>
{
    public static void InitializeService(DataServiceConfiguration config)
    {
        config.SetEntitySetAccessRule("*", EntitySetRights.All);
        config.SetServiceActionAccessRule("*", ServiceActionRights.Invoke);
        config.DataServiceBehavior.AcceptProjectionRequests = true;
        config.UseVerboseErrors = true;
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3;
    }

    protected override DynamicDbContext CreateDataSource()
    {
        var result = base.CreateDataSource();
        var dcf = new DynamicClassFactory();

        var context = new DynamicEntities();
        var templates = (from t in context.DynamicTemplates.Include("DynamicTemplateAttributes").Include("DynamicTemplateAttributes.DynamicAttribute")
                            select t);

        foreach (var dynamicTemplate in templates)
        {
            var type = CreateType(dcf, dynamicTemplate.Name, dynamicTemplate.DynamicTemplateAttributes);
            result.AddTable(type);
        }

        return result;
    }

    private Type CreateType(DynamicClassFactory dcf, string name, ICollection<DynamicTemplateAttribute> dynamicAttributes)
    {
        var props = dynamicAttributes.ToDictionary(da => da.DynamicAttribute.Name, da => typeof(string));
        var t = dcf.CreateDynamicType<BaseDynamicEntity>(name, props);
        return t;
    }

    [WebInvoke(Method = "POST")]
    public void CreateTable(string template)
    {
        var context = new DynamicEntities();
        var qry = (from dt in context.DynamicTemplates.Include("DynamicTemplateAttributes")
            .Include("DynamicTemplateAttributes.DynamicAttribute")
                    select dt).FirstOrDefault(dt => dt.Name == template);
        if (qry == null)
            throw new ArgumentException(string.Format("The template {0} does not exist", template));
        var ct = new StringBuilder();
        ct.AppendFormat("CREATE TABLE {0} (Id int IDENTITY(1,1) NOT NULL, ", qry.Name);
        foreach (var dta in qry.DynamicTemplateAttributes)
        {
            ct.AppendFormat("{0} nvarchar(255) NULL, ", dta.DynamicAttribute.Name);
        }
        ct.AppendFormat("CONSTRAINT [PK_{0}] PRIMARY KEY CLUSTERED", qry.Name);
        ct.AppendFormat("(Id ASC) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF," +
                        "ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY]");
        var ts = ct.ToString();
        CurrentDataSource.Database.ExecuteSqlCommand(ts);
    }
}

Now add a ASP.Net web project and make sure to apply the NuGet EntityFramework provider to it, add a reference to the DynamicServerSide project and add an svc file.  Add the following to the svc markup:

<%@ ServiceHost Language="C#"
    Factory="System.ServiceModel.Activation.WebServiceHostFactory, System.ServiceModel.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
    Service="DynamicServerSide.DynamicODataService" %>

Add another svc for the DynamicMetadataService and add its associated markup:

<%@ ServiceHost Language="C#"
    Factory="System.ServiceModel.Activation.WebServiceHostFactory, System.ServiceModel.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
    Service="DynamicServerSide.DynamicMetadataService" %>

For simplicity reasons, I have kept the table in the same database, here are the connection strings:

<connectionStrings>
  <add name="DynamicEntities" connectionString="metadata=res://*/DynamicModel.csdl|res://*/DynamicModel.ssdl|res://*/DynamicModel.msl;provider=System.Data.SqlClient;provider connection string=&quot;data source=.;initial catalog=Dynamic;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework&quot;" providerName="System.Data.EntityClient" />
  <add name="DynamicDbContext" connectionString="data source=.;initial catalog=Dynamic;integrated security=True;MultipleActiveResultSets=True" providerName="System.Data.SqlClient" />
</connectionStrings>

Below is the actual metadata service:

And now for the fun part and can you guess it.... 

Hope you got it, yes, we have actually an "unknown dynamically generated" strongly typed object exposed through our WCF Data Service.  Now that is great, we are now able to actually grab the browser and run queries from the address bar … go shout from the roof tops J

Here is the service and data:

And finaly the JSON format retrieving the data by key field 3:

{ "odata.metadata": "http://localhost:60086/DynamicODataService.svc/$metadata#SomeTables/@Element", "Id": 3, "Firstname":"My","Surname":"Mother","Gender":"Female"}

NOW THIS IS REALLY SWEET

Finally, Client Side

How do we consume it via a client.  In essence you can add a reference to the deployed web service, remember that adding it in design time will not actually create the classes, you have to run the service to create the data source.  But we want to do it dynamically without ever having to change the client side code.  Well this is where a lot of work needs to be done. 

First we need to obviously add a service reference to the meta data, just add it as a normal WCF service reference.

Secondly we have to manually build the client side up for the dynamic OData service.

Add a reference to the DynamicClassGenerator project, so we can generate the types client side.

VERY IMPORTANT
Note the ResolveTypeFromName and ResolveNameFromType, this will actually change the local type to the server type and the other way around and not throw you a big fat exception of the type not being found.  The other 2 vital methods are LoadTypes and GetServiceQuery.  LoadTypes will actually create the types and the GetServiceQuery is to build the query instruction up for the server.  Notice that in LoadTypes, the assembly generated is saved, how you save it, depends on your business/roll out logic.  You can basically save it, and just load it up at runtime if there is no change on the server side.

public partial class ProxyODataService : DataServiceContext
{
    public ProxyODataService(Uri serviceRoot)
        : base(serviceRoot, global::System.Data.Services.Common.DataServiceProtocolVersion.V3)
    {
        this.ResolveName = new global::System.Func<global::System.Type, string>(this.ResolveNameFromType);
        this.ResolveType = new global::System.Func<string, global::System.Type>(this.ResolveTypeFromName);
        this.OnContextCreated();
    }

    partial void OnContextCreated();
    protected string ResolveNameFromType(global::System.Type clientType)
    {
        if (clientType.Namespace.Equals("Dynamic.Objects", global::System.StringComparison.Ordinal))
        {
            return string.Concat("DynamicServerSide.", clientType.Name);
        }
        return null;
    }
    protected global::System.Type ResolveTypeFromName(string typeName)
    {
        global::System.Type resolvedType = this.DefaultResolveType(typeName, "DynamicServerSide", "Dynamic.Objects");
        if ((resolvedType != null))
        {
            return resolvedType;
        }
        return null;
    }

    public void LoadTypes(IEnumerable<DynamicTemplate> templates)
    {
        var dcf = new DynamicClassFactory();
        Types = new Dictionary<string, Type>();

        foreach (var dynamicTemplate in templates)
        {
            var type = CreateType(dcf, dynamicTemplate.Name, dynamicTemplate.DynamicTemplateAttributes);
            Types.Add(type.Name, type);
        }

        dcf.SaveAssembly();
    }

    public DataServiceQuery GetServiceQuery(string name)
    {
        var odd = typeof(ProxyODataService).GetMethod("CreateQuery");
        var mi = odd.MakeGenericMethod(Types[name]);
        var qr = mi.Invoke(this, new[] { name + "s" }) as DataServiceQuery; //assumes plural
        return qr;
    }

    private static Type CreateType(DynamicClassFactory dcf, string name, DataServiceCollection<DynamicTemplateAttribute> dynamicAttributes)
    {
        var props = dynamicAttributes.ToDictionary(da => da.DynamicAttribute.Name, da => typeof(string));
        var type = dcf.CreateDynamicType<BaseDynamicEntity>(name, props);
        return type;
    }

    public Dictionary<string, Type> Types { get; private set; }
}

To use the service client side, we have to use one more library I have stumbled across, called Dynamic Linq Library version 1.1.13 which is obtainable from Nuget by Nathan Arnott from Microsoft.  This allows us various extra capabilities of calling the WCF service by passing strings. 

With WPF/Silverlight, we have a bit of an advantage, the bindings are made real simple, so for a grid, all you need to do is call the web service and bring back a dynamic quearyable collection.

I have attached a small ViewModel, nothing fancy and not adhering to principles (Commands, etc) to just host the data and bind it to the Grid.  But the client side use is simply

var meta = new DynamicEntities(new Uri("http://localhost:60086/DynamicMetadataService.svc"));
var templates = (from t in meta.DynamicTemplates.Expand("DynamicTemplateAttributes/DynamicAttribute")
                    select t);

var odata = new ProxyODataService(new Uri("http://localhost:60086/DynamicODataService.svc"));
odata.LoadTypes(templates);


var qry = odata.GetServiceQuery("SomeTable");
ViewModel.Data = new ObservableCollection<dynamic>(qry.AsQueryable().ToDynamicArray());

Here is a sample result:

You can also run statements such as qry.Where("Firstname=\"Igor\""); where you could have had predefined lookup strategies, or filter type conditions.

I have been able to save data back to the database on changed properties, but it involves manually the object state in the context, eg: odata.ChangeState(entity, EntityStates.Modified); via odata.SaveChanges();

The reason for this is that we lost the ChangeTracker, might need to do some implementation myself.  At least the class have the property change notification that I could hook onto to do any further plumbing.

Additional work, will now include creating edit screens etc dynamically, but that can be stored in meta data etc, at least the data transport is basically resolved.

In the download you will find the code project and the database, it is for sql server 2014, but I have included scripts (table and data) you can run and just test it yourself, just create a database called Dynamic, otherwise you need to change the connection strings

Please note that I did test recreating the database and the code and all is working fine!

Building the solution will ensure that the sample will get the assemblies from NuGet.  Ensure you get the one from github.

PS: if you want to ask why I did not use a No-SQL database.  Well I have not had the time, and do not have the time to research that as well, especially on writing the various services and hooks for it, unless it exist out of the box.

Any comments are appreciated, especially around some fine tuning etc. and/or how to get the missing components working.

References:
These might or might not have been used, but gave insight and might still find their way into the code base, I might have forgotten to add 1 or 2, but will add them as I find them.

http://www.codeproject.com/Articles/121568/Dynamic-Type-Using-Reflection-Emit
http://www.codeproject.com/Articles/13337/Introduction-to-Creating-Dynamic-Types-with-Reflec
http://msdn.microsoft.com/en-us/library/3y322t50.aspx
http://www.codeproject.com/Articles/19513/Dynamic-But-Fast-The-Tale-of-Three-Monkeys-A-Wolf
https://grahammurray.wordpress.com/tag/reflection-emit/
http://www.codewrecks.com/blog/index.php/2008/08/04/implement-inotifypropertychanged-with-dynamic-code-generation/
http://blog.magnusmontin.net/2013/05/30/generic-dal-using-entity-framework/
http://www.codeproject.com/Articles/131587/Building-Linq-Expressions-Dynamically
http://corememorydump.blogspot.com/2012/02/wcf-data-services-reflection-custom.html
http://blogs.msdn.com/b/alexj/archive/2010/01/04/creating-a-data-service-provider-part-1-intro.aspx
http://stackoverflow.com/questions/12525528/share-poco-types-between-wcf-data-service-and-client-generated-by-add-service-re
https://www.nuget.org/packages/System.Linq.Dynamic.Library/1.1.13
http://dynamiclinq.azurewebsites.net/
http://www.remondo.net/repository-pattern-example-csharp/
http://blog.tonysneed.com/2013/11/18/trackable-entities-versus-self-tracking-entities/

AND THE ALL IMPORTANT MagicDbModelBuilder (for me that was)

https://github.com/maxbeaudoin/MagicDbModelBuilder

Points of Interest

This was a fair amount of effort, so I wish to share it with my fellow audience.