Click here to Skip to main content
15,607,524 members
Articles / Database Development / NoSQL
Posted 7 Feb 2019


9 bookmarked

Cachalot DB - Very Fast Transactional Database for .NET Applications - Part 2

Rate me:
Please Sign up or sign in to vote.
4.96/5 (7 votes)
16 Apr 2019CPOL4 min read
More on data manipulation


This is the second part of a series concerning Cachalot DB. The first part can be found here.

Compressing Object Data

The business objects are stored internally in a type-agnostic format. Index fields are stored as Int64 or string, and all the object data is stored as UTF-8 encoded JSON. The process of transforming a .NET object in the internal format is called “packing”. Packing is done client-side, the server only uses the indexes and manipulates the object as row data. It has no dependency on the concrete .NET datatype.

By default, the object data is not compressed but for objects that take more than a few kilobytes, compression may be very useful. For an object that takes 10 KB in JSON, compression ratio is around 1:10.

To enable compression, add a single attribute on the business data type.

public class Home


Using compressed objects is transparent for the client code. However, it has an impact on the packing time which is done on the client. When objects are retrieved, they are unpacked (which may imply decompression).

As a conclusion, compression may be very useful starting with medium size objects if you are ready to pay a small price, client-side only, for data insertion and retrieval.

Storing Polymorphic Collections in the Database

Polymorphic collections are natively managed. Type information is stored internally in the JSON and it is used to deserialize the proper concrete type.

A small example from a trading system:

Image 1

In order to store a collection of events, we must expose all required indexes on the base type.

Null values are perfectly acceptable for index fields which allow to expose indexed properties which make sense only for a specific child type.

public abstract class ProductEvent
                public int Id { get; set; }

                public abstract string EventType { get; }

                [Index(KeyDataType.IntKey, ordered:true)]
                public DateTime EventDate { get; set; }

                [Index(KeyDataType.IntKey, ordered: true)]
                public DateTime ValueDate { get; set; }      


public abstract class NegotiatedProductEvent: ProductEvent

public class IncreaseDecrease : NegotiatedProductEvent
                 public override string EventType => "IncreaseDecrease";

This is an example of code which retrieves a collection of concrete events from a DataStore typed with an abstract base class.

var events = connector.DataSource<ProductEvent>();

var increaseEvents = events.Where(

         evt => evt.EventType == "IncreaseDecrease" &&

         evt.EventDate == DateTime.Today


Conditional Operations and “Optimistic Synchronization”

A normal “put” operation adds an object or updates an existing one using the primary key as object identity.

More advanced use cases are implemented:

  1. Add an object only if it is not already there and tell me if it was really added
  2. Update an existent object only if the current version in the database satisfies a condition

The first one is available through the TryAdd operation on the DataSource class. If the object was already there, it is not modified, and it returns false. The test on the object existence and the insertion are executed as an atomic operation. The object cannot be updated or deleted by another client in-between.

That can be useful for data initialization, creating singleton objects, distributed locks, etc.

The second use case is especially useful for, but not limited to, the implementation of “optimistic synchronization”. The UpdateIf method on the DataSource class implements it.

If we need to be sure that nobody else modified an object while we were editing it (manually or algorithmically), there are two possibilities:

  • Lock the object during the edit operation. This is not the best option for a modern distributed system. A distributed lock is not suitable for massively parallel processing and if it is not released automatically (due to client or network failure), manual intervention by an administrator is required.
  • Use “optimistic synchronization”: do not lock but require that, when saving the modified object, the one in the database did not change since it was loaded. Otherwise, the operation fails, and we must retry (load + edit + save). This can be achieved in different ways:
    • Having a version on an object. When we save version n+1, we require that the object in the database is still at version n. In Cachalot DB, the syntax is items.UpdateIf(item, i=> i.Version == n-1)
    • Having a timestamp on an object. When we save a modified object, we require that the timestamp of the version in the database is identical to the one of the object before our update.
var oldTimestamp = item.Timestamp;
item.Timestamp = DateTime.Now;
items.UpdateIf(item, i=> i.Timestamp == oldTimestamp);

This can be even more useful when committing multiple object modifications in a transaction. If a condition is not satisfied on one object, rollback the whole transaction.

More on transaction in Part 3 ...

The fully open source code is available at:

Precompiled binaries and full documentation are available at:

The client code is available as nuget package at

To install: Install-Package Cachalot.Client


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Written By
Europe Europe
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

QuestionThanks Pin
Kuber Vats18-Apr-19 20:43
professionalKuber Vats18-Apr-19 20:43 
Questionany example to share? Pin
Southmountain17-Apr-19 16:46
Southmountain17-Apr-19 16:46 
AnswerRe: any example to share? Pin
Dan Ionescu (USINESOFT)18-Apr-19 9:27
Dan Ionescu (USINESOFT)18-Apr-19 9:27 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.