Big Design Up Front or Emergent Design?

Introduction

The question posed by this article is whether to develop a piece of software in a manner that can be categorised as Big Design Up Front or Emergent Design? This leads to the question of why not both? This article posits a set of tactics for achieving both by round tripping, but first, it is useful to have some discussion of architecture.

As can be seen from the picture above when drawing diagrams for layered and tiered architectures as boxes and arrows, the resulting pictures look the same. The difference being that in the first case the arrows represent dependencies whereas in the second, they represent dynamic links. For this article, the discussion will be limited to considering dependency analysis; performed at a low level, this typically produces spaghetti diagrams. To link the high (architectural) level with the low level of classes or files, a hierarchy will be used. This hierarchy can be formed from namespaces for classes or a (pseudo or actual) directory structure for files. It is useful at this point to introduce a metric for the sake of objectivity - the Cyclomatic Number of a graph is defined as E + P - N in terms of the number of edges (E), parts (P) and nodes (N). This then leads to a categorization of architecture patterns in terms of the metrics.

• Cyclomatic Number is zero: Can be represented as a set of trees. e.g. layered, hub and spoke.
• Low and graph without cycles: The usual case, e.g., the so-called layered architecture with cross-cutting concerns.
• Otherwise: Big ball of mud.

It should be noted that this is quite assertive on the existence of cycles, such bad practices are promoted by the GoF in the Design Patterns book. For example, see the C++ source for the visitor pattern where the visitor and visitee classes reference each other. This cycle may be broken by defining the visitee in two parts with the second part only adding the visit function and the visitor only referencing the first part.

Tools

For drawing the graph of dependencies, this article will use the tool DeepEnds (Visual Studio extension / NuGet package) to produce DGML files for manipulation within Visual Studio. Note that the tool also reports the Cyclomatic Number, and more, for viewing in a web browser. There are alternative tools for producing DGML files such as those included within Visual Studio Enterprise, there are also other extensions listed on Visual Studio Gallery for manipulating DGML. The command line, rather than the GUI, version of DeepEnds will be used. The recommended choice for processing a particular language are:

• C#: Supply csproj / sln files as input to the Roslyn based parser
• Visual Basic: Supply vbproj / sln files as input to the Roslyn based parser
• .NET: Supply DLL / EXE files as input to the Mono.Cecil based decompiler
• C++: Supply vcxproj / sln files for the optional libclang based parser
• C: Supply vcxproj / sln files for the default parser that reads source code for #include statements and vcxproj.filter files for the hierarchy
• Others: Supply the path to a Doxygen XML output file (which may instead have been generated by a bespoke tool)

The resulting DGML file may then be manipulated within Visual Studio; nodes can be moved between groups and the edges representing the dependencies between the groups will be re-calculated as can be seen in the following series of screen shots.

YAGCI?

The reason why Big Design Up Front is frowned upon is that requirements change over the development life cycle leading to rework of both the code and the documentation. This problem can be ameliorated by generating the documentation from source code, where the as-is portion is taken from the working code and the to-be part can be sketched out with stub code. The stub code would probably not be produced if the development is too assertive in applying the concepts of YAGNI and MVP, which leads to the question of You Aren't Gonna Consider It?

One question leads to another, in this case of what then to consider? It helps to refer to the concept of Software Product Lines where components are combined to form different products for different customers. If this is not obvious, then the reader should at least be familiar with producing test suites for internal QA in addition to a product for paying customers. It is proposed that the designer uses stub code to produce a virtual Product Line by considering the known concrete functionality and then, by a process of adding and subtracting pieces, create a set of potential products. By only including code with relevant functionality to a particular product then modularity can be driven through the design of the codebase potentially reducing the cost of an unplanned pivot.

To support this process, each product should have a DGML file generated for it and the system as a whole should be built to ensure the stub code can be correctly parsed. The former can be achieved by using a batch file to run DeepEnds repeatedly with different arguments. The resulting diagrams can then be individually manipulated to split nodes representing concrete code into a more flexible structure.

Refactoring

Once the DGML files have been modified, they are no longer live documents and the source code has to be refactored to bring the two back into sync. Considered, from the viewpoint of Lean development, refactoring is rework and thus waste that should be minimised. The point of stubbing the code is an attempt to minimise architectural refactoring, however that still leaves the issue of how to minimise the (hopefully lesser) refactoring work generated by the process.

• ReSharper provides the ability to move C# types to different files and namespaces
• Namespace statements in C++ header and source files should be concatenated on the same line to allow for easier modification due to not having to change the indentation
  • namespace A { namespace B {
  • namespace A::B { when C++ 17 proposal N4230 becomes supported
• Changing filters in a Visual Studio project can be as easy as drag and drop

Discussion

Having defined the architecture for an application, it is possible to use static analysis, potentially as part of the build process, to enforce the allowed dependencies. For C# projects, one possibility is NsDepCop, an example rule from its XML configuration file is <Allowed From="*" To "System.*" />. In Visual Studio 2017, Microsoft is introducing a rule system called layer validation.

Hopefully, this article has demonstrated to the reader that they don't have to choose between Big Design Up Front and Emergent Design. That tools exists to plot a middle way and thus to reduce the chance of having to enter refactoring hell. Obviously, full round-tripping can only be achieved if altering the graphs directly alters the code. Implementing such a tool is an exercise left to the reader...

Further Reading

• Dive into Architecture with DeepEnds
• Dependency Analysis in Visual C++
• Dependency Analysis with Doxygen
• Why Favour the Cyclomatic Number?

History

• 2016/11/28: First release