Scripted Business Rules with .NET and IronPython, Part 1

• Download demo - 2.4 MB

Introduction - What are Business Rules?

Added Part 2 which has all of the unit test code, plus a full WPF app.

You’ve probably encountered this before. You’ve deployed the new version of your system, and now the sales team has a hot new lead, Acme Corp. They send you this email:

“Would it be possible, when the user creates a new Product inventory record, to automatically set the serial number to the Division abbreviation, followed by a dash, then a unique six digit number? Oh, and they need to add a -01 suffix if it’s a boat.”

Well, now what?

What if you could just browse to your web app, log in as system admin, and write something like this?

import clr
clr.AddReference('Sigma.Data.Helpers')
from Sigma.Data.Helpers import *

##
## Helper method
##
def assignSN(prod, prod_type):
    div = prod_type.CompanyDivision
    num = Atomic.GetNextNumber()
    suffix = ''
    if prod_type.Name.lower() == 'boat':
        suffix = '-01'
    sn = '{div}-{num}{suffix}'.format(
        div = div.Abbrev,
        num = str(num).zfill(6),
        suffix = suffix)
    prod.SerialNumber = sn

##
## Saving a product inventory record, need to set S/N
## according to Acme Corp. rules.
##
def onInform(sender, e):
    prod = sender
    if e.Info == 'Saving':
        sn = prod.SerialNumber
        prod_type = prod.ProductType
        if prod_type and not sn:
            assignSN(prod, prod_type)

What, not How

If you’re like most developers, the question that sprang to mind is, “How am I going to do that? The Serial Number field is just a string of text – I don’t know how to format it for every conceivable use case.”

Scripted business rules can step in and handle the “how”. Once you grasp this, the concept is incredibly freeing. You can focus your efforts on what your system needs to capture, what important events need to be raised, and the all-important relationships between objects.

Linus Torvalds, the creator of Linux, said this: “Bad programmers worry about the code. Good programmers worry about data structures and their relationships.” Once you see the dynamic power of business rules in action, you’ll see the wisdom of that quote. More importantly, you’ll have the tools there, ready to step in to take care of those pesky requirements, so that you can focus on the design of your system and the way things interact.

Business Rules Separate Data from Behavior

The process of assigning a serial number is well-known: Someone or something has created a string of digits and characters, and we need to store it. This is the data entry part of assigning a serial number. Most likely, this is baked in to your code.

The behavior is the bit that differs. When Acme Corp creates a product inventory record, their behavior says that the serial number must be in a specific format that contains a prefix and an atomically-incrementing number.

Business rules allow you to focus on fundamental processes in your baked-in code. The connection to behavior becomes as simple as having your baked-in, compiled code say “here is some information about something that is about to happen or just happened.”

Formal Definition of Business Rules

A business rule is a declaration about some aspect of a company’s day-to-day requirements. Many texts on business rules get very academic about things like constraints, derivations, facts, assertions, etc. While those are useful concepts, they are also pretty boring. We’d rather get the declarations out of the way and start writing code!

Some texts on business rules are too broad. You’ll see examples like “An Employee may be assigned to many Teams” as a “business rule”. Well – yes. But you’ve already covered that when you designed your system. That’s really more of what we call a “baked-in” rule: You aren’t going to change that requirement for Acme Corp. but leave it in place for Widgets Inc.

Example: A Business Rule Discovered by Requirements Gathering

When a shipment is marked as received, we must send an email with the corresponding PO number to the accounting department so they can enter it into MRP.

Here we have several “things” and several “actions”: shipment, marked, received, send, email, purchase order, accounting, enter, MRP.

Business rule declarations can help you spot gaps in your processes. For example, if your system currently has no way to track a shipment, it’s going to need that functionality.

In this case, something else comes out: we have a “why” statement at the end: “so they can enter it into MRP”. “Why” statements can be important to identify possible future enhancements, or to allow the software team to suggest alternatives. For example, the development team may suggest directly modifying the MRP system so that accounting doesn’t have to hand-copy data from an email.

Our Definition of Business Rules

So that's the formal definition. Pretty dry and boring, and not much practical use when we just need to get something done. Here's our definition for the sake of this article series:

We want to keep our definition short, but a more specific and expanded version reads like this: “A business rule is a scripted event handler or script triggered by a user-initiated command which can be redefined at runtime within the system itself, to change the behavior of the system.”

Background - How it was Developed

Many years ago, I developed a product configuration system. The sheer amount of variability within that system was nearly overwhelming. It was an online tool that was used to configure everything from boats to firefighter clothing to medical devices. Each of my customers had vast numbers of rules about the way things interacted: this option is only available with these other two options, this feature depends on that feature, etc.

I developed a pretty robust "declarative rules system" that could handle about 80% of the complexity. But we all know the 80/20 rule, right? Yep - I spent way too much time trying to overcome that 20% that just didn't follow the rules.

The code that I'll present in this series grew out of that 20%: The only way to handle it was with code which was specific to each of my customers. As with most subsystems, this one grew from handling specific needs to become more general-purpose as I understood the phenomenal power of what I'd discovered.

I can confidently report that as of today, it can be used in nearly any .NET-based system, whether desktop or web. It can be used in multi-tenant systems. And, it has been completely rewritten to remove every vestige of its "product configuration" roots.

A very similar (but older) branch of this code is currently deployed in several web applications. The exact same DLLs are also used in an extremely high-throughput assembly line optical inspection system. My experience in developing all of these systems can vouch for the wide-ranging applicability of a system like this.

Image: Notification maintenance business rule defined in multi-tenant web application. Each tenant has different rules about how to handle marking and deleting notifications.

Image: File system and database maintenance business rule defined in high-throughput WPF inspection system. Each of this customer's locations around the world has different rules about how much data, and what types of data, can be stored before automated cleanup tasks kick in.

The Goal

The goal of this project was simple - make business rules definable as simply another data model class that's stored in your data store along with products, customers, sales, etc. They are edited, saved, and retrieved just like any other domain entity.

How it Works

The Aim.Scripting library (included as a DLL in the download) defines an implementation of a special interface called IRuntimeExecuter. This class, DefaultExecuter, handles two tasks: connecting to standard .NET events, and allowing the execution of arbitrary commands.

Triggering a script-connected event handler from C# code is dead simple. It requires one line of code that looks like this:

public event EventHandler MyEvent;

// This is the script-connected handler signature for events that
// are directly defined in this class.
protected virtual void OnMyEvent(EventArgs e)
{
    e.FireWithScriptListeners(() => MyEvent, this);
}

Or like this:

// Here we are overriding a handler. The MyEvent event is defined in
// the base class. Note the slightly different signature of the
// extension method call.
protected override void OnMyEvent(EventArgs e)
{
    e.FireWithScriptListeners(base.OnMyEvent, this);
}

You don't even need to perform a null check on e and the event - the extension method will do it for you.

The IronPython script method that handles this event is even simpler:

def onMyEvent(sender, e):
    pass

These Python scripts are stored in a class that implements the IScriptDefinition interface. Don't worry - it's super easy to implement and the solution in the download (and the code below) shows exactly how it's done. The IScriptDefinition interface defines the name of the module, the text of the scripts, and a "type key" that indicates what kinds of types and events the scripts should listen for. In short, it's all standard textual data that's really simple to save into whatever data store you choose.

The Python signature is always on (lowercase), followed by the event name, then sender, e as the argument list. So a .NET event called SomethingHappened would be connected to a Python signature of:

def onSomethingHappened(sender, e):
    ## put code here to handle the event
    pass

If you need a Python primer, I've posted one here: Python Primer. The Official Python Documentation is excellent and very deep. You'll also want to learn as much as you can about .NET and Python integration, which is what IronPython is all about. For that, head on over to IronPython on CodePlex.

Here is the entire definition of BizRuleDataModel, which is our implementation of IScriptDefinition in the test project attached to this article.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Aim;
using Aim.Scripting;

namespace BizRules.DataModels
{
    /// <summary>
    /// The IScriptDefinition implementation for our domain. This is a
    /// data model class just like Product, Customer, etc.
    /// </summary>
    [
    Serializable()
    ]
    public partial class BizRuleDataModel : DomainBase, IScriptDefinition
    {
        private string m_Name;
        private string m_Scripts;
        private string m_TypeKey;
        private ICollection<BizRuleCmdDataModel> m_Commands =
            new List<BizRuleCmdDataModel>();

        /// <summary>
        /// Convenience method to find a script definition by its name.
        /// </summary>
        /// <param name="moduleName"></param>
        /// <returns></returns>
        public static BizRuleDataModel FindByModuleName( string moduleName )
        {
            return RepositoryProvider.Current
                .GetRepository<BizRuleDataModel>().GetAll()
                .FirstOrDefault( x => x.Name == moduleName );
        }

        /// <summary>
        /// The stored name. This is returned by GetModuleName().
        /// </summary>
        public string Name
        {
            get
            {
                return Get( m_Name );
            }
            set
            {
                Set( ref m_Name, value, "Name" );
            }
        }

        /// <summary>
        /// The stored Python scripts. This is returned by GetScripts().
        /// </summary>
        public string Scripts
        {
            get
            {
                return Get( m_Scripts );
            }
            set
            {
                Set( ref m_Scripts, value, "Scripts" );
            }
        }

        /// <summary>
        /// The stored type key. This is used to indicate what types this
        /// script definition should connect to.
        /// </summary>
        public string TypeKey
        {
            get
            {
                return Get( m_TypeKey );
            }
            set
            {
                Set( ref m_TypeKey, value, "TypeKey" );
            }
        }

        /// <summary>
        /// A collection of predefined commands that can map to functions
        /// defined in the scripts. These can be shown in a drop-down list
        /// for user selection. When the user selects a command, we can
        /// pop up a "parameter collector" window that collects argument
        /// information from the user, then when they press the "OK" or
        /// "Execute" button, we can call the Python script with the collected
        /// parameters.
        /// </summary>
        public ICollection<BizRuleCmdDataModel> Commands
        {
            get
            {
                return Get( m_Commands );
            }
            set
            {
                Set( ref m_Commands, value, "Commands" );
            }
        }

        /// <summary>
        /// Overridden. When this script definition changes, we need to
        /// notify the ScriptFactory.
        /// </summary>
        public override void Refresh()
        {
            ScriptFactory.Current.Invalidate( this );
        }

        #region IScriptDefinition Members...

        /// <summary>
        /// Returns the Name property.
        /// </summary>
        /// <returns></returns>
        public string GetModuleName()
        {
            return Name;
        }

        /// <summary>
        /// Returns the Scripts property.
        /// </summary>
        /// <returns></returns>
        public string GetScripts()
        {
            return Scripts;
        }

        /// <summary>
        /// Returns the TypeKey property.
        /// </summary>
        /// <returns></returns>
        public string GetTypeKey()
        {
            return TypeKey;
        }
        #endregion ...IScriptDefinition Members
    }
}

As you can see, the implementation of IScriptDefinition is simply a matter of forwarding three string properties.

The whole system needs two more things to enable basic functionality. The Aim.Scripting.ScriptFactory needs to know where to find the script definitions that it will use. For that, we need to inherit from ScriptProviderBase and override a single method:

using System;
using System.Collections.Generic;
using Aim.Scripting;

namespace BizRules.DataModels.Tests
{
    /// <summary>
    /// Our script provider implementation.
    /// </summary>
    public class ScriptProvider : ScriptProviderBase
    {
        /// <summary>
        /// This is the only method that must be overridden. The script factory must know
        /// where to find the script definitions that it will use.
        /// </summary>
        /// <returns></returns>
        public override IEnumerable<IScriptDefinition> GetAllDefinitions()
        {
            return RepositoryProvider.Current
                .GetRepository<BizRuleDataModel>().GetAll();
        }
    }
}

Now that we have a ScriptProvider, we need to register it with ScriptFactory. This can be done anywhere in your "composition root" (which in the case of our project is the Setup method of the test classes).

[TestInitialize]
public void Setup()
{
    //
    // Init script factory
    //
    ScriptFactory.Current.Initialize( () => new ScriptProvider() );
    
    // ... other init things
}

So, in sum, to get everything working, we need:

• Classes that raise "script-connected" events with one of the FireWithScriptListeners extension methods. These can be any .NET class that you define or can inherit from.
• A persistent (data model, domain model, entity, active record, etc.) class that implements the IScriptDefinition interface and will be stored along with other data in the domain.
• A ScriptProviderBase-derived class that overrides the GetAllDefinitions() method. This class should be defined in or close to your composition root; that is, it should probably be defined in the UI project of your solution, whether it's a console, web, WPF, forms, or whatever.
• A line of code in your "composition root" (test setup method, web Global.asax, WPF bootstrapper, etc) class that will register your script provider implementation with ScriptFactory.
• And, of course, some actual Python code stored in your implementation of IScriptDefinition.

Using the code

The code download is a solution with three projects:

• A project that defines "data models" and "repositories". This project defines things like BizRuleDataModel, ProductDataModel, CustomerDataModel, etc. - all the classes that are used as a data store for the Unit Test project.
  • Look closely at the ListRepository<T> class that's defined here. That's where you will see events being raised on behalf of other objects via the RaiseInform(...) method.
• A "service" project with a phony, do-nothing EmailingProvider ambient context implementation.
• Finally, the actual Unit Test project.

Here is the code from EventTests.cs.

using System;
using System.Linq;
using Aim;
using Aim.Scripting;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace BizRules.DataModels.Tests
{
    /// <summary>
    /// These tests have to do with "script-connected events".
    /// </summary>
    [TestClass]
    public class EventTests
    {
        private const string PROMO_CODE = "JULY";

        /// <summary>
        /// Init the script factory, load the repos with some test data.
        /// </summary>
        [TestInitialize]
        public void Setup()
        {
            //
            // Init script factory
            //
            ScriptFactory.Current.Initialize( () => new ScriptProvider() );
            //
            // Save some stuff to the "database"
            //
            var rp = RepositoryProvider.Current;
            //
            // Create some products
            //
            var pRepo = rp.GetRepository<ProductDataModel>();
            var p = pRepo.Create();
            p.Name = "Marbles";
            p.Price = 1m;
            pRepo.Save( p );
            p = pRepo.Create();
            p.Name = "Jacks";
            p.Price = 10m;
            pRepo.Save( p );
            p = pRepo.Create();
            p.Name = "Spam";
            p.Price = 5m;
            pRepo.Save( p );
            p = pRepo.Create();
            p.Name = "Eggs";
            p.Price = 4.50m;
            pRepo.Save( p );
            //
            // Create some business rules
            //
            var brRepo = rp.GetRepository<BizRuleDataModel>();
            //------------------------
            var br = brRepo.Create();
            br.Name = "Forwarded Events";
            br.TypeKey = ScriptFactory.Current.CreateTypeKey( typeof( ForwardedEvents ) );
            br.Scripts =
@"

##
## Got an Inform event via ForwardedEvents. e.Item will have a reference
## to the forwarded object that we want to work with.
##
def onInform(sender, e):
    e.Item.Something = e.Info

";
            brRepo.Save( br );
            //------------------------
            br = brRepo.Create();
            br.Name = "Sale Events";
            br.TypeKey = ScriptFactory.Current.CreateTypeKey( typeof( SaleDataModel ) );
            br.Scripts =
@"

##
## Something happened. Set the Notes property so we know
## this was called.
##
def onSomeEvent(sender, e):
    sender.Notes = 'Hello'

##
## Give everyone the promo, we're feeling generous
##
def onPropertyChanged(sender, e):
    if e.PropertyName == 'SaleNumber':
        sender.PromoCode = 'JULY'

##
## Do the JULY promo, 10% off
##
def onInform(sender, e):
    if e.Info == 'Saving':
        if sender.PromoCode and sender.PromoCode.lower() == 'july':
            for item in sender.LineItems:
                product = item.Product
                if product:
                    item.Price = product.Price * 0.9
        else:
            for item in sender.LineItems:
                product = item.Product
                if product:
                    item.Price = product.Price

##
## Do something that is impossible without script handlers!
##
def onDeserialized(sender, e):
    sender.PromoCode = None

";
            brRepo.Save( br );
        }

        /// <summary>
        /// This test demonstrates the concept of "event forwarding". This is
        /// a pattern you can use when you need to raise events about something,
        /// but you don't have access to the code for the object, or you cannot
        /// derive from the class.
        /// </summary>
        [TestMethod]
        public void PocoCanHookToScriptEventsViaForwarding()
        {
            string info = "Hello";
            var p = new PocoDataModel();
            var events = new ForwardedEvents();
            events.RaiseInform( p, info );

            // =====
            // Event handler in biz rule should have set property on
            // the PocoDataModel.
            // =====
            Assert.IsNotNull( p.Something );

            // =====
            // Should have set the Something property on the PocoDataModel
            // to the Info string of the event args (Hello).
            // =====
            Assert.AreEqual( p.Something, info );
        }

        /// <summary>
        /// The FireWithScriptListeners extension methods should properly
        /// connect, call the scripted handler, and then disconnect the event,
        /// all in one line of code.
        /// </summary>
        [TestMethod]
        public void DynamicHandlersAreProperlyRemoved()
        {
            var sale = RepositoryProvider.Current.GetRepository<SaleDataModel>().Create();
            sale.RaiseSomeEvent();

            // =====
            // The script should have set the Notes property to a non-null value
            // =====
            Assert.IsNotNull( sale.Notes );

            // =====
            // Executer should have successfully disconnected after running event
            // =====
            Assert.AreEqual( sale.GetSomeEventHandlerCount(), 0 );
        }

        /// <summary>
        /// Can we write a script to handle the PropertyChanged event?
        /// </summary>
        [TestMethod]
        public void ScriptRecognizesPropertyChange()
        {
            var sale = RepositoryProvider.Current.GetRepository<SaleDataModel>().Create();
            sale.SaleNumber = "123";

            // =====
            // Script handler should have set the promo code
            // =====
            Assert.AreEqual( sale.PromoCode, PROMO_CODE );
        }

        /// <summary>
        /// Test something a little more complicated: Set pricing in a Sale
        /// object based on a promo code.
        /// </summary>
        [TestMethod]
        public void SavingSaleSetsPromoPricing()
        {
            var saleRepo = RepositoryProvider.Current.GetRepository<SaleDataModel>();
            var sale = saleRepo.Create();
            sale.SaleNumber = "456";

            // =====
            // Script handler should have set the promo code
            // =====
            Assert.AreEqual( sale.PromoCode, PROMO_CODE );

            //
            // Create a couple line items
            //
            var pRepo = RepositoryProvider.Current.GetRepository<ProductDataModel>();
            var marbles = pRepo.GetAll().FirstOrDefault( x => x.Name == "Marbles" );
            var jacks = pRepo.GetAll().FirstOrDefault( x => x.Name == "Jacks" );
            var spRepo = RepositoryProvider.Current.GetRepository<SaleProductDataModel>();
            var lineItem = spRepo.Create();
            lineItem.SaleId = sale.Id;
            lineItem.ProductId = marbles.Id;
            lineItem.Sequence = 0;
            sale.LineItems.Add( lineItem );
            lineItem = spRepo.Create();
            lineItem.SaleId = sale.Id;
            lineItem.ProductId = jacks.Id;
            lineItem.Sequence = 1;
            sale.LineItems.Add( lineItem );
            saleRepo.Save( sale );

            var productsPrice = marbles.Price + jacks.Price;

            // =====
            // Based on the promo code, the Inform saving event should have
            // set the price of each line item to 0.9 * product price.
            // =====
            Assert.AreEqual( productsPrice * 0.9m, sale.TotalPrice );

            sale.PromoCode = "NONE";
            saleRepo.Save( sale );

            // =====
            // Not a known promo code, should reset the price to the normal
            // product price.
            // =====
            Assert.AreEqual( productsPrice, sale.TotalPrice );
        }

        /// <summary>
        /// This is a fun one. Aim.NotifierBase defines a Deserialized
        /// event - which would normally be useless. But not with scripted
        /// handlers. Because scripted handlers connect, run, disconnect,
        /// they can be "connected" to objects that don't exist yet!
        /// </summary>
        [TestMethod]
        public void SomethingImpossibleLikeDeserializationEvents()
        {
            var saleRepo = RepositoryProvider.Current.GetRepository<SaleDataModel>();
            var sale = saleRepo.Create();
            sale.SaleNumber = "789";

            // =====
            // Script set the promo code, as per usual.
            // =====
            Assert.IsTrue( sale.PromoCode.IsNotNil() );

            var saleCopy = BinarySerializer.Copy( sale );

            // =====
            // We connected to an "impossible to connect to" event
            // handler that nulled out the promo code!
            // =====
            Assert.IsTrue( saleCopy.PromoCode.IsNil() );
        }
    }
}

Unzip the solution (don't forget to "unblock" the zip file before you unzip it and copy to your final destination folder).

Open the Aim.Scripting.Demo solution. I use NuGet "package restore", so if you get errors about missing IronPython (the only outside dependency), you'll need to run:

PM> install-package ironpython

from the Package Manager Console in Visual Studio, with Default Project set to BizRules.DataModels.Tests.

The code was developed in VS 2013, but it targets .NET Framework 4.0, so it should open in anything from VS 2010 forward.

To run the code once you have the solution open, choose Test --> Run --> All Tests.

Spend time looking at the code in the tests - that's why it's commented. Try creating your own data models, business rules models, etc. Add your own test class and use the provided ones as a guide. Create your own classes with your own events and implement FireWithScriptListeners as shown here. Go crazy!

What are These Things Good For?

• Custom communications
  • Implement a real EmailingProvider. I recommend Mandrill. The Mandrill service (mandrillapp.com) is wonderful, and this wrapper code makes it much easier to work with.
• Custom logging
• Custom data validation
• Custom data formatting (as in our opening Serial Number example)
• Custom field semantics
  • Harness the power of "custom fields" in your database, by using business rules to give them actual meaning. Calculated fields? No big deal - just look for a ValueChanged event on a couple different custom fields, then set the value of the calculated field when one of the others changes. No more stupid little editors with weird syntax rules and indecipherable drag-and-drop operators, and "formulas" scattered everywhere in the system.
• Custom, object-based security
• Custom maintenance
• Custom reports
  • Commands can return any object. So that includes HTML, JSON, a list of entities, a DataSet...
• Custom, event-based archival tasks
  • def onBeforeDelete(sender, e): ...
• Forward integration (event-based)
  • Reference your customer's ERP-vendor API DLL (good luck with that...), and push data to ERP whenever an event says you saved a new sale, customer service call, etc. Or better yet, just push it to a custom data store that you've set up just for that customer, and let them deal with the extraction.
• Reverse integration (command-based)
  • Provide a list of commands on the Products page of your site to pull certain information from your ERP system (again, God be with you...)
• Product configuration and Fact Bases (of course)
• Global or tenant-specific settings

And the list goes on...

We're Just Getting Started

This is a subject that I'm passionate about. I've invested about eight years of my life, off and on, to what I'm now putting up here. I've found it to be a total game-changer. I now think about where in my C# code I might want to notify about an interesting happening, rather than how I'm going to deal with that interesting happening. That concept, once it sinks in, will hopefully result in the same eureka moment for you.

In future installments, we'll cover those "what's it good for" list items in much more detail. Also, I hope to make part 2 an actual running application that stores the data for real. Not sure if it will be a web or WPF app at this point.

History

First draft completed 2015/05/29.

Added Part 2.