Databases Do Not Matter and Other Things Which Will Stop Mattering

Introduction

This article is a gist of my understanding of how application design has evolved over the years. In this discussion, I will try to highlight the design trends over the decades and where I believe architecture and design styles will move us in future. “I am not trying to bash DB Designers here.”

How We Started...

I remember talking to some of my teachers who used to code in the days when you had to make your own storage systems. People used to say “If you get you data structures right, then rest will follow”. We also used to argue on various algorithms for mundane tasks like sorting and searching which now are taken for granted because languages just take care of them. The important point here is that we concentrated upon how information was represented as a point of our primary focus and its manipulation as a secondary. Let’s call this as Gen 1 design.

Let’s fast forward to the time of having out of the box databases (may be they were RDBMS or Network databases – i.e. things like Foxpro for the really young people). At this time, we probably picked up a really bad habit, everything was designed around the databases and structures which were manipulated by application were modeled behind the database and somehow managed to tie up to the presentation layer. At this point of time, due to operational issues, more and more logic started getting built around database and business logic was being written in stored procedure so that we could get maximum work done in one DB call. The focus was not on how data is being represented in the persistence store. Let’s call this Gen 2 design.

Where We Are Today...

Let’s fast forward when OO world came, with languages like Java and .NET (which can’t match the performance of a native code). We still carried some of the bad habits of Gen 2 designs. We still focused on DB driven design, still wrote a lot of code in stored procedures (sometimes because an old system was being migrated to new stack and we wanted to protect investment). The point is we were in an OO world, we had a so called layered architecture, but we were being driven by RDBMS tables to do our design. I have worked with architects who only focused on DB design and let things flow from there. Now to look at it in principle. Gen 1 and Gen 2 designs were the same, being driven by data representation mechanism and not OO principles.

This philosophy of Data first approach has led to the following problems:

1. Business layers in countless applications is a thin wrapper to Data access while the business logic is written in extremely tough to debug stored procedures.
2. RDBMSs do not support the concepts of various OO principles like inheritance or composition, etc. which lead to having very flat data transfer objects (also known as value objects) and in principle we did not graduate ourselves to thinking in objects.
3. We are duplicating our work, we spend a lot of time creating a database and then duplicate and identical amount of effort in creating our value objects and keeping them in sync is a pain.

What the Future Will Be Like

In my humble opinion in future databases will not matter, we will not have to explicitly design a database. It will just store our objects, in other words I believe the world will be an OO database with object just being stored and their storage mechanism not being defined.

In my opinion, architectures of the future will be somewhat as under:

The core of any business will be driven by an object model. This will define business entities that exist and will be fairly static. We will also add the basic CRUD behavior to these, however the difference is that the Datastore (not database) will be just another library where you CRUD the entities. No special things will be required to be designed.

In the component of object core, the creational patterns will flourish. The primary responsibility of this layer will be the ability to represent the business objects, create them and rely upon a persistence store.

It is highly probable that industries may standardize these object models just the way XML structures for such a large number of industries have been standardized.

On top of this layer will be a wrapping service interface where business rules specific to a given business will be written. This will be the domain of behavior patterns, this will be done to ensure that object model will not be tightly coupled to business logic. In this place, various extensions to the object core will be made custom tailored to business needs.

This is the place where use cases will be exposed as service interfaces. What is important is that the service interface will be just another library for its consumers.

Components like UI, background services and others will be using these libraries.

Why Databases Do Not Matter Will Come First?

Let’s look at the trend of the industry the thing we know most stops mattering, thirty years ago the big debate was “Which sorting algorithm should I use?” The debate today is how do I integrate with other systems, sure we have come up with many XML schemas, but we have not technically matured to figure out the best way to do this.

Databases have somehow evolved into being much more than data store. They are starting to govern application design and development (due to performance), as an industry we have realized this and will need to stop to have applications with loose coupling.

Databases are a duplicate component which should not matter and steps like OODBMS will ensure that they will not matter.

What Else Won't Matter?

Eventually other things will stop mattering. These will be, How do I integrate? Probably a mutated from REST exposing business entities as CRUD will be the way to go. I support REST because I see it as just another redundant element, its CRUD exposed over another dimension (not database) and what we learn making Databases just a framework will be applicable to this mutated form of REST.

Information technology consists of three things:

1. Entities – These will be defined by the industry, they will eventually get standardized.
2. CRUD – This is a simple piece of technology, call it OODBMS or call it REST or some other mechanism. All we are looking at is storage and retrieval of data. If paper doesn’t need to change for an industry, why should it matter for computers?
3. Business Rules – This will be the area where IT will matter in customization of this rule set for people.

History

• 25^th July, 2010: Initial post