Learn EventSourceDB through Unit Testing

• Source Codes at GitHub
• Tag0505

Introduction

About Event Sourcing and EventStoreDB, there are plenty of white papers and tutorials around. Presuming that you have walked through those, this article presents some integration tests to illustrate the runtime behaviours of a raw event sourcing persistent store like EventStoreDb, providing some wet materials for studying.

References:

• EventStoreDb: Getting Started

Background

The first time I had heard of something similar to event sourcing was "Big Data" and Google search DB in late 90s, which allow only data appending, no deletion or update upon physical data storage, for good business cases: performance and scaling, and good business contexts: the abundance of digital storage. 

EventStoreDB or alike is not something you as a programmer can't live without, however, it may make you live more comfortably falling into the Pit of Success, especially for "12 Transformative Benefits of Event Sourcing".

Using the code

Firstly go to EventStoreDB's developer portal, and install a local DB server and .NET client API (v24.2 as of May 2024). After the installation, you should be able to see the following:

The First Test

/// <summary>
/// Basic example from EventSourceDb tutorial on https://developers.eventstore.com/clients/grpc/#creating-an-event
/// </summary>
/// <returns></returns>
[Fact]
public async Task TestBasic()
{
    var evt = new TestEvent
    {
        EntityId = Guid.NewGuid(),
        ImportantData = "I wrote my first event!"
    };

    var eventData = new EventData(
        Uuid.NewUuid(),
        "TestEvent",
        JsonSerializer.SerializeToUtf8Bytes(evt)
    );

    const string connectionString = "esdb://admin:changeit@localhost:2113?tls=true&tlsVerifyCert=false";
    /// tls should be set to true. Different from the official tutorial as of 2024-05-05 on https://developers.eventstore.com/clients/grpc/#creating-an-event. 
    /// I used the zipped EventStoreDb installed in Windows machine, launched with `EventStore.ClusterNode.exe --dev`

    var settings = EventStoreClientSettings.Create(connectionString);
    using EventStoreClient client = new(settings);
    string streamName = "some-stream";
    IWriteResult writeResult = await client.AppendToStreamAsync(
        streamName,
        StreamState.Any,
        new[] { eventData }
        );

    EventStoreClient.ReadStreamResult readStreamResult = client.ReadStreamAsync(
        Direction.Forwards,
        streamName,
        StreamPosition.Start,
        10);

    ResolvedEvent[] events = await readStreamResult.ToArrayAsync();
    string eventText = System.Text.Encoding.Default.GetString(events[0].Event.Data.ToArray());
    TestEvent eventObj = JsonSerializer.Deserialize<TestEvent>(eventText);
    Assert.Equal("I wrote my first event!", eventObj.ImportantData);
}

As you can see, the official tutorial was not working out of the box, at least with the EventSourceDb Windows release. So having the first test with R/W operations, I have become confident in exploring more.

Remarks/Hints:

• The test cases/facts presented in this article are not really of unit testing, as I am just using the test suites built on a unit testing framework to explorer the functionality of EventSourceDB during learning. However, such test suite could be useful in a commercial app development since you could have a simple testbed to explore and debug when something in your app codes has gone wrong and the respective call stack may involve external dependency like EventSourceDb.

The First Test Suite

Since EventSourceDb v21, the API protocol had changed from "RESTful" Web API to gRPC. And it is generally recommended to reuse a gRPC connection. Therefore, EventStoreClientFixture for IClassFixture is crafted.

public class EventStoreClientFixture : IDisposable
{
    // todo: What is the best way to clear an event store for unit tests? https://github.com/EventStore/EventStore/issues/1328
    public EventStoreClientFixture()
    {
        IConfigurationRoot config = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();
        string eventStoreConnectionString = config.GetConnectionString("eventStoreConnection");
        var settings = EventStoreClientSettings.Create(eventStoreConnectionString);
        Client = new(settings);
    }

    public EventStoreClient Client { get; private set; }

    #region IDisposable pattern
    bool disposed = false;

    void Dispose(bool disposing)
    {
        if (!disposed)
        {
            if (disposing)
            {
                Client.Dispose();
            }

            disposed = true;
        }
    }

    public void Dispose()
    {
        Dispose(true);
    }
    #endregion
}

The fixture codes will be executed by xUnit.NET runtime once for each test class inherited from IClassFixture<EventStoreClientFixture>, therefore test cases in the same class may share the same EventStoreClient or gRPC connection. 

However, please keep in mind, with xUnit, each test/fact is executed in a new instance of the test class, that is, if a test class contains 10 facts, the class may be instantiated 10 times.

public class BasicFacts : IClassFixture<EventStoreClientFixture>
{
    public BasicFacts(EventStoreClientFixture fixture)
    {
        eventStoreClient = fixture.Client; // all tests here shared the same client connection
    }

    readonly EventStoreClient eventStoreClient;

    [Fact]
    public async Task TestBackwardFromEnd()
    {
        string importantData = "I wrote my test with fixture " + DateTime.Now.ToString("yyMMddHHmmssfff");
        var evt = new TestEvent
        {
            EntityId = Guid.NewGuid(),
            ImportantData = importantData,
        };

        var eventData = new EventData(
            Uuid.NewUuid(),
            "testEvent", //The name of the event type. It is strongly recommended that these use lowerCamelCase, if projections are to be used.
            JsonSerializer.SerializeToUtf8Bytes(evt), // The raw bytes of the event data.
            null, // The raw bytes of the event metadata.
            "application/json" // The Content-Type of the EventStore.Client.EventData.Data. Valid values are 'application/json' and 'application/octet-stream'.
        );

        string streamName = "some-stream2";
        IWriteResult writeResult = await eventStoreClient.AppendToStreamAsync(
            streamName,
            StreamState.Any,
            new[] { eventData }
            );

        EventStoreClient.ReadStreamResult readStreamResult = eventStoreClient.ReadStreamAsync(
            Direction.Backwards,
            streamName,
            StreamPosition.End,
            10);
        Assert.Equal(TaskStatus.WaitingForActivation, readStreamResult.ReadState.Status);

        ResolvedEvent[] events = await readStreamResult.ToArrayAsync();
        string eventText = System.Text.Encoding.Default.GetString(events[0].Event.Data.ToArray());
        TestEvent eventObj = JsonSerializer.Deserialize<TestEvent>(eventText);
        Assert.Equal(importantData, eventObj.ImportantData); // so the first in events returned is the latest in the DB side.
    }
...
...
...

As you can see from the run, the initial connection to a local hosted EventSourceDb server may take over 2 seconds, and the subsequent calls are fast.

Test Negative Cases

For real world applications, having the codes compiled and having the functional features working are far from enough from delivering business values to the customers. It is important to proactively test negative cases for the sake of defensive programming and setup basic defence lines for delivering decent User Experience. 

/// <summary>
/// Test with a host not existing
/// https://learn.microsoft.com/en-us/aspnet/core/grpc/deadlines-cancellation
/// </summary>
/// <returns></returns>
[Fact]
public async Task TestUnavailableThrows()
{
    var evt = new TestEvent
    {
        EntityId = Guid.NewGuid(),
        ImportantData = "I wrote my first event!"
    };

    var eventData = new EventData(
        Uuid.NewUuid(),
        "TestEvent",
        JsonSerializer.SerializeToUtf8Bytes(evt)
    );

    const string connectionString = "esdb://admin:changeit@localhost:2000?tls=true&tlsVerifyCert=false"; // this connection is not there on port 2000
    var settings = EventStoreClientSettings.Create(connectionString);
    using EventStoreClient client = new(settings);
    string streamName = "some-stream";
    var ex = await Assert.ThrowsAsync<Grpc.Core.RpcException>(() => client.AppendToStreamAsync(
        streamName,
        StreamState.Any,
        new[] { eventData }
        ));
    Assert.Equal(Grpc.Core.StatusCode.Unavailable, ex.StatusCode);
}

/// <summary>
/// Simulate a slow or disrupted connection to trigger error.
/// </summary>
/// <returns></returns>
[Fact]
public async Task TestDeadlineExceededThrows()
{
    var evt = new TestEvent
    {
        EntityId = Guid.NewGuid(),
        ImportantData = "I wrote my first event!"
    };

    var eventData = new EventData(
        Uuid.NewUuid(),
        "TestEvent",
        JsonSerializer.SerializeToUtf8Bytes(evt)
    );

    string streamName = "some-stream";
    var ex = await Assert.ThrowsAsync<Grpc.Core.RpcException>(() => eventStoreClient.AppendToStreamAsync(
        streamName,
        StreamState.Any,
        new[] { eventData },
        null,
        TimeSpan.FromMicroseconds(2) // set deadline very short to trigger DeadlineExceeded. This could happen due to network latency or TCP/IP's nasty nature.
        ));
    Assert.Equal(Grpc.Core.StatusCode.DeadlineExceeded, ex.StatusCode);
}

/// <summary>
/// 
/// </summary>
/// <returns></returns>
[Fact]
public async Task TestReadNotExistingThrows()
{
    EventStoreClient.ReadStreamResult readStreamResult = eventStoreClient.ReadStreamAsync(
        Direction.Backwards,
        "NotExistingStream",
        StreamPosition.End,
        10);
    Assert.Equal(TaskStatus.WaitingForActivation, readStreamResult.ReadState.Status);

    var ex = await Assert.ThrowsAsync<EventStore.Client.StreamNotFoundException>(async () => { var rs = await readStreamResult.ToArrayAsync(); });
    Assert.Contains("not found", ex.Message);
}

Getting familiar with such error scenarios will help you to apply defensive programming in your app codes and design fault tolerance overall.

Stress Tests

Surely a unit testing framework is not a very good testbed for stress testing or benchmarking, however, it could be good enough to obtain rough idea about how fast a "fact" could run. So if the same "fact" become dramatically slower, it may signal something wrong in the evolving implementation of some functions.

The following facts are not really unit testing or integration testing to explore the basic performance of EventSourceDB.

[Fact]
public async Task TestBackwardFromEndWriteOnly_100()
{
    for (int i = 0; i < 100; i++)
    {
        string importantData = "I wrote my test with fixture " + DateTime.Now.ToString("yyMMddHHmmssfff");
        var evt = new TestEvent
        {
            EntityId = Guid.NewGuid(),
            ImportantData = importantData,
        };

        var eventData = new EventData(
            Uuid.NewUuid(),
            "testEventStress",
            JsonSerializer.SerializeToUtf8Bytes(evt),
            null,
            "application/json"
        );

        string streamName = "some-streamStress";
        IWriteResult writeResult = await eventStoreClient.AppendToStreamAsync(
            streamName,
            StreamState.Any,
            new[] { eventData }
            );

        Assert.True(writeResult.LogPosition.CommitPosition > 0); 
    }
}

Remarks:

• If some CI/CD pipeline is involved, you may want to exclude these test cases from the pipeline.
   

Points of Interest

EventStoreDb utilizes long (64-bit signed) and ulong (64-bit unsigned), it will be interesting to see how a JavaScript client could handle and overcome the 53-bit precision limit. Please stay tuned, as in next few weeks I may be adding a few more articles about Event Sourcing and EventStoreDb:

• TypeScript Codes Talking to EventSourceDb
• Audit Trail through Event Sourcing
• Time Travel through Event Sourcing

These articles will be using integration test suites to guard the app codes and illustrate the runtime behaviours of functional features involving EventSourceDB. 

References:

• The Good, the Bad and the Ugly: How to choose an Event Store?
• EventStoreDB VS Kafka