Introduction
Over the last year and a half, I have developed MyXaml into what I think is a fairly mature generic XML instantiation engine, I've written a "lite" version called MycroXaml, developed a simple utility that takes XML and generates the underlying imperative classes and properties suitable for instantiation by the declarative parser, and explored the strengths and weaknesses of declarative programming. One of the tools that was requested by many people, but missing from this suite, was the ability to take the declarative code and generate imperative code. In order to address that need, this article presents an initial cut at an XML compiler. The work that I'm presenting here takes MyXaml styled XML and, using CodeDom, generates the imperative code that instantiates the XML object graph.
If you are unfamiliar with declarative programming concepts, I recommend you read my article "Comparing Declarative And Imperative Code" as an introduction to the differences in the two programming styles.
Now to confuse everyone. While MyXaml and MycroXaml probably have a bit of brand name recognition, I'm going to slowly migrate away from the term "xaml". You will instead begin to see files and XML nodes using the term "declara". I've decided to go this route because the generic instantiation engines and supporting tools that I've written, and will continue to write, are moving in a very different direction than Microsoft's XAML, and frankly, I don't want to have my tools confused with Microsoft's. There. How's that for spin-doctoring!
What about Mono?
Iain McCoy has been implementing a XAML compiler as part of Google's "Summer of Code" program. You can read a little bit about it here or just Google for "XAML compiler". There are several significant differences between what I've done and what Iain appears to be working towards:
- I'm not interested in following the Microsoft's XAML namespace syntax.
- I'm not interested in following the implicit collection syntax (note the
IAddChild interface). All of my parsers strictly follow a "class-property-class" hierarchy. - I wanted the resulting code to be "human readable". To some extent, this will get me in a bit of trouble if two different namespaces have the same class name. This will be resolved later when I refactor the code.
- Along with being human readable, I also wanted property conversions from
strings to look natural and be as efficient as possible. So rather than w.SecondsToDestruction = (new Int32Converter()).ConvertFromString("5"); and similar constructs for things like Point, Size, and Font converters, I've implemented a general purpose mechanism for emitting the appropriate CodeDom, so, for example, the resulting code will look like m.SecondsToDestruction=5;. The point of compiled code is to remove all the type conversion, right? On the other hand, this means that there are structs and other classes that I'm not currently supporting, but are definitely easy to add.
So, that pretty much summarizes the differences between what Iain is doing and what I'm doing. As you can see, there's some justification for my reasoning to disassociate myself from XAML.
Without further ado...
An XML Compiler
The XML compiler uses the code originally described in the MycroXaml article, but it has evolved a bit. I chose this code base because I wanted a simpler prototyping environment for the first pass of the compiler. The original code base has been modified to fire a variety of events during parsing:
public event InstantiateClassDlgt InstantiateClass;
public event AssignPropertyDlgt AssignProperty;
public event AssignEventDlgt AssignEvent;
public event SupportInitializeDlgt BeginInitCheck;
public event SupportInitializeDlgt EndInitCheck;
public event EventHandler EndChildProcessing;
public event AddToCollectionDlgt AddToCollection;
public event UseReferenceDlgt UseReference;
public event AssignReferenceDlgt AssignReference;
public event CommentDlgt Comment;
The compiler hooks these events and instantiates various CodeDom statements. You can still use the parser to instantiate an object graph at runtime, or you can use the CodeGen tool to generate the C#, VB, or other CodeDom emitable code and compile and assemble. The other significant change to the parser is that when it is in the "code generation" mode, nothing is being instantiated and no properties are being assigned, so the code had to be made more robust to handle this condition.
Unit Tests
First off, I needed to write a variety of unit tests to verify the basic functionality of the parser and code generator:
These are pseudo unit tests because they don't actually test the resulting source code, they only test that the parser and the code generator don't find a fault in the process of compiling the XML. So, for example, the "SimpleClass" unit test:
[Test]
public void SimpleClass()
{
string xml="<?xml version='1.0' encoding='utf-8'?>\r\n";
xml+="<!-- (c) 2005 MyXaml All Rights Reserved -->\r\n";
xml+="<Declara Name='CodeGenTest'\r\n";
xml+=" xmlns:def='Definition'\r\n";
xml+=" xmlns:ref='Reference'\r\n";
xml+=" xmlns:wf='System.Windows.Forms, System.Windows.Forms,
Version=1.0.5000.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089'>\r\n";
xml+=" <wf:Form Name='appMainForm'/>";
xml+="</Declara>\r\n";
XmlDocument doc=new XmlDocument();
doc.LoadXml(xml);
CodeGen cg=new CodeGen(doc, "CodeGenTest",
"Clifton.CodeGenTest", "CodeGenTest");
}
invokes the code generator and emits the following code:
namespace Clifton.CodeGenTest
{
using System.Windows.Forms;
public class CodeGenTest
{
private Form appMainForm;
public CodeGenTest()
{
}
public virtual object Initialize()
{
appMainForm = new Form();
appMainForm.SuspendLayout();
appMainForm.Name = "appMainForm";
appMainForm.ResumeLayout();
return this.appMainForm;
}
}
}
The Ultimate Test Case
The final test case that I used is the example of the color chooser:
This applet involves several controls, data binding, and wiring up event handlers. The complete markup appears as follows:
="1.0"="utf-8"
<Declara Name="Form"
xmlns:wf="System.Windows.Forms, System.Windows.Forms,
Version=1.0.5000.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089"
xmlns:ctd="Clifton.Tools.Data, Clifton.Tools.Data"
xmlns:ev="Events, Events"
xmlns:def="Definitions"
xmlns:ref="References">
<wf:Form Name="appMainForm"
Text="Color Chooser"
ClientSize="400, 190"
BackColor="White"
FormBorderStyle="FixedSingle"
StartPosition="CenterScreen">
<ev:TrackbarEvents def:Name="events"/>
<wf:Controls>
<wf:TrackBar Name="redScroll" Orientation="Vertical"
TickFrequency="16" TickStyle="BottomRight" Minimum="0"
Maximum="255" Value="128"
Scroll="{events.OnScrolled}" Size="42, 128"
Location="10, 30" Tag="R"/>
<wf:TrackBar Name="greenScroll" Orientation="Vertical"
TickFrequency="16" TickStyle="BottomRight" Minimum="0"
Maximum="255" Value="128"
Scroll="{events.OnScrolled}" Size="42, 128"
Location="55, 30" Tag="G"/>
<wf:TrackBar Name="blueScroll" Orientation="Vertical"
TickFrequency="16" TickStyle="BottomRight" Minimum="0"
Maximum="255" Value="128"
Scroll="{events.OnScrolled}" Size="42, 128"
Location="100, 30" Tag="B"/>
<wf:Label Size="40,15" TextAlign="TopCenter"
Font="Microsoft Sans Serif, 8.25pt, style= Bold"
Location="10, 10" ForeColor="Red" Text="Red"/>
<wf:Label Size="40,15" TextAlign="TopCenter"
Font="Microsoft Sans Serif, 8.25pt, style= Bold"
Location="55, 10" ForeColor="Green" Text="Green"/>
<wf:Label Size="40,15" TextAlign="TopCenter"
Font="Microsoft Sans Serif, 8.25pt, style= Bold"
Location="100, 10" ForeColor="Blue" Text="Blue"/>
<wf:Label Name="redValue" Size="40,15" TextAlign="TopCenter"
Font="Microsoft Sans Serif, 8.25pt, style= Bold"
Location="10, 160" ForeColor="Red" Text="128">
</wf:Label>
<wf:Label Name="greenValue" Size="40,15" TextAlign="TopCenter"
Font="Microsoft Sans Serif, 8.25pt, style= Bold"
Location="55, 160" ForeColor="Green" Text="128">
</wf:Label>
<wf:Label Name="blueValue" Size="40,15" TextAlign="TopCenter"
Font="Microsoft Sans Serif, 8.25pt, style= Bold"
Location="100, 160" ForeColor="Blue" Text="128">
</wf:Label>
<wf:PictureBox Name="colorPanel" Location="90, 0" Size="200, 100"
Dock="Right" BorderStyle="Fixed3D" BackColor="128, 128, 128"/>
</wf:Controls>
<ev:TrackbarEvents ref:Name="events"
ColorPanel="{colorPanel}"/>
<ctd:BindHelper Source="{redScroll}" SourceProperty="Value"
Destination="{redValue}" DestinationProperty="Text"
ImmediateMode="true"/>
<ctd:BindHelper Source="{greenScroll}" SourceProperty="Value"
Destination="{greenValue}" DestinationProperty="Text"
ImmediateMode="true"/>
<ctd:BindHelper Source="{blueScroll}" SourceProperty="Value"
Destination="{blueValue}" DestinationProperty="Text"
ImmediateMode="true"/>
</wf:Form>
</Declara>
Data Binding
One of the things I've discovered about declarative programming is that I prefer to keep the UI object graph separate from the data binding (and, actually, even the event wire-ups). The binding I'm using here is not .NET's data binding; instead, I'm using the code described in my article Understanding Simple Data Binding. While not technically necessary, it's more convenient since .NET doesn't have a default constructor for the Binding class, which makes it unsuitable for declarative instantiation without a wrapper.
The Event Handler
The event handler is placed in its own assembly. Essentially, this is how you would code the application specific implementation for UI and other events - an assembly (or assemblies) holds the imperative code and the declarative code maps the namespace and instantiates the classes. In the above XML, you will notice that the TrackbarEvents class is instantiated early in the process so that the TrackBar.Scroll event can be wired to the OnScrolled event handler. Later on, the TrackbarEvents instance is referenced so that the PictureBox instance can be assigned to it. The reason for all of this is that the MycroParser doesn't handle forward references. The event handler itself is straightforward, combining the R, G, and B channels into a single Color structure and assigning it as the background color:
using System;
using System.Collections;
using System.Drawing;
using System.Windows.Forms;
namespace Events
{
public class TrackbarEvents
{
protected byte[] channels=new byte[3] {128, 128, 128};
protected Hashtable channelMap;
protected PictureBox colorPanel;
public PictureBox ColorPanel
{
get {return colorPanel;}
set {colorPanel=value;}
}
public TrackbarEvents()
{
channelMap=new Hashtable();
channelMap["R"]=0;
channelMap["G"]=1;
channelMap["B"]=2;
}
public void OnScrolled(object sender, EventArgs e)
{
TrackBar track=(TrackBar)sender;
int idx=(int)channelMap[track.Tag];
channels[idx]=(byte)track.Value;
colorPanel.BackColor = Color.FromArgb(channels[0],
channels[1], channels[2]);
}
}
}
The resulting "compiled" C# code looks like this (I've snipped out the redundant portions of the code):
namespace Clifton.CodeGenTest
{
using System.Windows.Forms;
using Clifton.Tools.Data;
using Events;
using System.ComponentModel;
using System.Drawing;
public class CodeGenTest
{
private Form appMainForm;
private TrackbarEvents events;
private TrackBar redScroll;
private TrackBar greenScroll;
private TrackBar blueScroll;
private Label label1;
private Label label2;
private Label label3;
private Label redValue;
private Label greenValue;
private Label blueValue;
private PictureBox colorPanel;
private BindHelper bindHelper1;
private BindHelper bindHelper2;
private BindHelper bindHelper3;
public CodeGenTest()
{
}
public virtual object Initialize()
{
appMainForm = new Form();
appMainForm.SuspendLayout();
events = new TrackbarEvents();
redScroll = new TrackBar();
redScroll.BeginInit();
redScroll.SuspendLayout();
redScroll.Name = "redScroll";
redScroll.Orientation = Orientation.Vertical;
redScroll.TickFrequency = 16;
redScroll.TickStyle = TickStyle.BottomRight;
redScroll.Minimum = 0;
redScroll.Maximum = 255;
redScroll.Value = 128;
redScroll.Scroll +=
new System.EventHandler(events.OnScrolled);
redScroll.Size = new Size(42, 128);
redScroll.Location = new Point(10, 30);
redScroll.Tag = "R";
redScroll.EndInit();
redScroll.ResumeLayout();
appMainForm.Controls.Add(redScroll);
... snipped green and blue ...
label1 = new Label();
label1.SuspendLayout();
label1.Size = new Size(40, 15);
label1.TextAlign = ContentAlignment.TopCenter;
label1.Font = new Font("Microsoft Sans Serif",
8, FontStyle.Bold);
label1.Location = new Point(10, 10);
label1.ForeColor = Color.Red;
label1.Text = "Red";
label1.ResumeLayout();
appMainForm.Controls.Add(label1);
... snipped the other two labels ...
redValue = new Label();
redValue.SuspendLayout();
redValue.Name = "redValue";
redValue.Size = new Size(40, 15);
redValue.TextAlign = ContentAlignment.TopCenter;
redValue.Font = new Font("Microsoft Sans Serif",
8, FontStyle.Bold);
redValue.Location = new Point(10, 160);
redValue.ForeColor = Color.Red;
redValue.Text = "128";
redValue.ResumeLayout();
appMainForm.Controls.Add(redValue);
... snipped the green and blue value labels ...
colorPanel = new PictureBox();
colorPanel.SuspendLayout();
colorPanel.Name = "colorPanel";
colorPanel.Location = new Point(90, 0);
colorPanel.Size = new Size(200, 100);
colorPanel.Dock = DockStyle.Right;
colorPanel.BorderStyle = BorderStyle.Fixed3D;
colorPanel.BackColor = Color.FromArgb(128, 128, 128);
colorPanel.ResumeLayout();
appMainForm.Controls.Add(colorPanel);
events.ColorPanel = colorPanel;
bindHelper1 = new BindHelper();
bindHelper1.BeginInit();
bindHelper1.Source = redScroll;
bindHelper1.SourceProperty = "Value";
bindHelper1.Destination = redValue;
bindHelper1.DestinationProperty = "Text";
bindHelper1.ImmediateMode = true;
bindHelper1.EndInit();
... snipped the other two binder setups ...
appMainForm.Name = "appMainForm";
appMainForm.Text = "Color Chooser";
appMainForm.ClientSize = new Size(400, 190);
appMainForm.BackColor = Color.White;
appMainForm.FormBorderStyle =
FormBorderStyle.FixedSingle;
appMainForm.StartPosition =
FormStartPosition.CenterScreen;
appMainForm.ResumeLayout();
return this.appMainForm;
}
}
}
One of the nifty features is that comments in your XML code are placed as comments in the compiled source code!
Execution
The "CompileAndRun" unit test creates the assembly, instantiates the class, invokes the Initialize method, and then performs a ShowDialog on the resulting Form instance:
[Test]
public void CompileAndRun()
{
XmlDocument doc=new XmlDocument();
doc.Load(
"..\\..\\Demos\\MycroParser\\bin\\debug\\ColorPicker.declara");
CodeGen cg=new CodeGen(doc, "Form",
"Clifton.CodeGenTest", "CodeGenTest");
string source=cg.Source;
RunTimeCompiler rtc=new RunTimeCompiler();
Assembly assembly=rtc.Compile(cg.Assemblies, "C#",
source, String.Empty,
Assembly.GetExecutingAssembly().Location);
object refObj=Activator.CreateInstance(
assembly.GetModules(false)[0].GetTypes()[0]);
object ret=refObj.GetType().InvokeMember("Initialize",
BindingFlags.Public | BindingFlags.Instance |
BindingFlags.InvokeMethod, null, refObj, null);
ret.GetType().InvokeMember("ShowDialog", BindingFlags.Public |
BindingFlags.Instance | BindingFlags.InvokeMethod,
null, ret, null);
}
I'm not going to go into the details of the RunTimeCompiler class, as that is off-topic.
CodeDom
Working with the CodeDom is interesting, so I'm going to illustrate snippets from the various parser event handlers.
Instantiating the Wrapper Class
The wrapper class holds the fields, default class constructor, and Initialize method for the compiled code. In order to construct this CodeDom unit, the CodeDom has to be initialized:
provider=new CSharpCodeProvider();
gen=provider.CreateGenerator();
followed by creating the code compile unit and adding the namespace unit:
CodeCompileUnit ccu=new CodeCompileUnit();
cns=new CodeNamespace(ns);
ccu.Namespaces.Add(cns);
then adding the class type declaration, code constructor, and Initialize method:
ctd=new CodeTypeDeclaration(className);
cns.Types.Add(ctd);
constructor=new CodeConstructor();
constructor.Attributes=MemberAttributes.Public;
ctd.Members.Add(constructor);
method=new CodeMemberMethod();
method.Name="Initialize";
method.ReturnType=new CodeTypeReference("System.Object");
method.Attributes=MemberAttributes.Public;
ctd.Members.Add(method);
Instantiating a Class
When an XML node is encountered, it is either a class instantiation or a class property. If it's a class instantiation, the generator has to create a field of that type:
CodeMemberField cmf=
new CodeMemberField(cea.Type.Name, name);
cmf.Attributes=MemberAttributes.Private;
ctd.Members.Add(cmf);
and then adding the appropriate "new" code assignment:
CodeAssignStatement instantiator=
new CodeAssignStatement(
new CodeVariableReferenceExpression(name),
new CodeObjectCreateExpression(cea.Type.Name,
new CodeExpression[] {}));
method.Statements.Add(instantiator);
Note that there are no parameters passed to the constructor.
Assigning a Property Value
Assigning property values is where a lot of magic takes place.
Assigning Enum Values
The enum assignment is a bit complicated, because some properties can take a set of enums that are binary OR'd together, such as the Anchor property. The following code illustrates how the CodeDom is generated to handle this case. The following code isn't 100% robust, but it's a good start:
protected CodeExpression EvalEnums(string itemStr,
string typeName)
{
string[] items=itemStr.Split(',');
string item=items[0];
CodeBinaryOperatorExpression expr2=null;
CodeExpression expr=new CodeFieldReferenceExpression(
new CodeTypeReferenceExpression(typeName),
item.Trim());
if (items.Length > 1)
{
expr2=new CodeBinaryOperatorExpression();
expr2.Operator=CodeBinaryOperatorType.BitwiseOr;
expr2.Left=expr;
expr=expr2;
}
for (int i=1; i<items.Length; i++)
{
CodeExpression right=new CodeFieldReferenceExpression(
new CodeTypeReferenceExpression(typeName),
items[i].Trim());
if (i+1 == items.Length)
{
expr2.Right=right;
}
else
{
CodeBinaryOperatorExpression b2=
new CodeBinaryOperatorExpression();
b2.Operator=CodeBinaryOperatorType.BitwiseOr;
expr2.Right=b2;
b2.Left=right;
expr2=b2;
}
}
return expr;
}
Assigning Values that Require Type Conversion
The next complicated thing to handle is assigning values, such as "400, 190" to structures such as Point and Size. I've chosen an extensible mechanism for this, so that the CodeDom can be generated that is specific to the type of property. For example:
[AssignType("System.Drawing.Point")]
protected CodeExpression EvalPoint(string val)
{
AddNamespace("System.Drawing");
AddAssembly("System.Drawing.dll");
string[] coord=val.Split(',');
return new CodeObjectCreateExpression("Point",
new CodeExpression[]
{
new CodePrimitiveExpression(Convert.ToInt32(coord[0].Trim())),
new CodePrimitiveExpression(Convert.ToInt32(coord[1].Trim())),
});
}
Again, not necessarily the most robust code in the world, but it gets the job done for now.
You will note the AssignType attribute that decorates the method. The code generator looks for methods decorated with an attribute identifying the property type, and if it finds an appropriate method, it will invoke it and allow the method to return a CodeExpression that can be used as the r-value of the assignment statement. This approach lets me tailor the resulting code to a well-optimized assignment. The drawback is that not all of the structs and classes that might need to be evaluated have been implemented.
Simple Types
Finally, value types that are not structures are converted from a string to the property type:
object cval=Converter.Convert(pea.Value,
pea.PropertyInfo.PropertyType);
assignVal=new CodePrimitiveExpression(cval);
Creating the Assignment Statement
And finally, the assignment statement is constructed:
CodeAssignStatement assign=
new CodeAssignStatement(
new CodePropertyReferenceExpression(
new CodeVariableReferenceExpression(currentMember),
pea.PropertyInfo.Name),
assignVal);
Event Assignment
Event assignment (associating the method of an instance to an event) is actually pretty straightforward:
private void OnAssignEvent(object sender, EventEventArgs eea)
{
CodeAttachEventStatement assign=new CodeAttachEventStatement(
new CodeEventReferenceExpression(
new CodeVariableReferenceExpression(currentMember),
eea.EventInfo.Name),
new CodeDelegateCreateExpression(
new CodeTypeReference(eea.EventInfo.EventHandlerType.FullName),
new CodeVariableReferenceExpression(eea.SourceName),
eea.MethodName));
method.Statements.Add(assign);
eea.Handled=true;
}
ISupportInitialize and Layout Support
Classes that implement ISupportInitialize should have BeginInit and EndInit called. Similarly, classes (all Control classes, for example) that implement SuspendLayout and ResumeLayout should also have these methods called so that these methods bracket any property assignments. This is handled by the following code, and illustrates method invocation in the CodeDom:
private void OnBeginInitCheck(object sender,
SupportInitializeEventArgs siea)
{
TypeFilter filter=new TypeFilter(InterfaceFilter);
Type[] interfaces=siea.Type.FindInterfaces(filter,
"System.ComponentModel.ISupportInitialize");
if (interfaces.Length > 0)
{
AddNamespace("System.ComponentModel");
AddAssembly("System.dll");
CodeMethodInvokeExpression cmie=
new CodeMethodInvokeExpression(
new CodeVariableReferenceExpression(currentMember),
"BeginInit", new CodeExpression[] {});
method.Statements.Add(cmie);
}
if (siea.Type.GetMethod("SuspendLayout") != null)
{
CodeMethodInvokeExpression cmie=
new CodeMethodInvokeExpression(
new CodeVariableReferenceExpression(currentMember),
"SuspendLayout", new CodeExpression[] {});
method.Statements.Add(cmie);
}
siea.Handled=true;
}
The OnEndInitCheck method is similar.
Adding to Collections
The code generator assumes that adding instances to properties implementing a collection (ICollection or IList) is accomplished using the Add method. A shortcut that I'm taking here is to assume that if the property is read-writable, then it is not a collection, but rather the current node is a specialized instance being assigned to the property of the grandparent node. If the property is read only, then the code generator assumes that the property is a collection. To make this more robust involves considerable work and is part of the MyXaml parser. The current implementation looks like this:
private void OnAddToCollection(object sender,
CollectionEventArgs cea)
{
string parentMember=(string)currentMemberStack.ToArray()
[currentMemberStack.Count-1];
if (cea.PropertyInfo.CanWrite)
{
CodeAssignStatement assign=new CodeAssignStatement(
new CodePropertyReferenceExpression(
new CodeVariableReferenceExpression(parentMember),
cea.PropertyInfo.Name),
new CodeVariableReferenceExpression(currentMember));
method.Statements.Add(assign);
}
else
{
CodeMethodInvokeExpression cmie=new CodeMethodInvokeExpression(
new CodePropertyReferenceExpression(
new CodeVariableReferenceExpression(parentMember),
cea.PropertyInfo.Name),
"Add",
new CodeExpression[]
{
new CodeVariableReferenceExpression(currentMember),
});
method.Statements.Add(cmie);
}
cea.Handled=true;
}
Comments
Adding a comment is trivial compared to some of the other processes:
private void OnComment(object sender, CommentEventArgs cea)
{
method.Statements.Add(new CodeSnippetStatement(""));
CodeCommentStatement ccs=new CodeCommentStatement(
new CodeComment(cea.Comment, false));
method.Statements.Add(ccs);
}
The Code
Unzip the download and navigate to the Clifton.Tools.Xml folder, in which you will find the Clifton.Tools.sln file, which is the solution you will want to open.
Conclusion
The ability to compile an XML object graph into your favorite .NET language opens a variety of doors. In many ways, it's easier to visualize the object graph in XML than it is in code because of the hierarchical layout of XML, and therefore it's also easier to make changes. XML is also language neutral, so you can write your object graphs declaratively and then work with the imperative code in whatever language you're comfortable with. It's also less verbose (imagine that!) than imperative code. However, the drawbacks with XML are numerous - Intellisense isn't readily available yet, along with syntax checking, and there's an "emotional" resistance that many people experience working with XML in a declarative way when all they've ever done is imperative coding. And of course, up to now, there have been the issues of performance and security, which are addressed by compiling the XML and generating an assembly and thus completely eliminating the parser. Hmmm. Wait a minute!
One Last Comment
If you're interested in contributing to this project, please leave a comment below, and I'll set you up with an account on the CVS repository.
