Click here to Skip to main content
15,919,613 members
Articles / Programming Languages / C#

Prefer using Stream to byte[]

Rate me:
Please Sign up or sign in to vote.
5.00/5 (28 votes)
12 Apr 2023CPOL2 min read 27.9K   30   25
A short example of how byte[] misuse may hurt memory consumption
While reviewing and refactoring a real-world codebase, I've noticed how byte[] API is misused. That is the reason why in this article I'm sharing some thoughts on why you shouldn't evade Stream API in your code.


When working with files, there are often both APIs operating byte[] and Stream so quite often people chose byte[] counterpart as it requires less ceremony or just intuitively more clear.

You may think of this conclusion as far-fetched but I’ve decided to write about it after reviewing and refactoring some real-world production code. So you may find this simple trick neglected in your codebase as some other simple things I’ve mentioned in my previous articles.


Let’s look at the example as simple as calculating file hash. In spite of its simplicity, some people believe that the only way to do it is to read the entire file into memory.

Experienced readers may have already foreseen a problem with such an approach. Let’s see do some benchmarking on 900MB file to see how the problem manifests and how we can circumvent it.

The baseline will be the naive solution of calculating hash from byte[] source:

public static Guid ComputeHash(byte[] data)
    using HashAlgorithm algorithm = MD5.Create();
    byte[] bytes = algorithm.ComputeHash(data);
    return new Guid(bytes);

So following the advice from the title of the article, we’ll add another method that will accept Stream convert it to byte array and calculate hash.

public async static Task<Guid> ComputeHash(Stream stream, CancellationToken token)
    var contents = await ConvertToBytes(stream, token);
    return ComputeHash(contents);

private static async Task<byte[]> ConvertToBytes(Stream stream, CancellationToken token)
    using var ms = new MemoryStream();
    await stream.CopyToAsync(ms, token);
    return ms.ToArray();

However, calculating hash from byte[] is not the only option. There’s also an overload that accepts Stream. Let’s use it.

public static Guid ComputeStream(Stream stream)
    using HashAlgorithm algorithm = MD5.Create();
    byte[] bytes = algorithm.ComputeHash(stream);
    stream.Seek(0, SeekOrigin.Begin);
    return new Guid(bytes);

The results are quite telling. Although execution time is pretty similar, memory allocation varies dramatically.

Image 1

So what happened here? Let’s have a look at ComputeHash implementation

public byte[] ComputeHash(Stream inputStream)
    if (_disposed)
        throw new ObjectDisposedException(null);
    // Use ArrayPool.Shared instead of CryptoPool because the array is passed out.
    byte[] buffer = ArrayPool<byte>.Shared.Rent(4096);
    int bytesRead;
    int clearLimit = 0;
    while ((bytesRead = inputStream.Read(buffer, 0, buffer.Length)) > 0)
        if (bytesRead > clearLimit)
            clearLimit = bytesRead;
        HashCore(buffer, 0, bytesRead);
    CryptographicOperations.ZeroMemory(buffer.AsSpan(0, clearLimit));
    ArrayPool<byte>.Shared.Return(buffer, clearArray: false);
    return CaptureHashCodeAndReinitialize();

While we tried to blindly follow the advice in the article, it didn’t help. The key takeaway from these figures is that using Stream allows us to process files in chunks instead of loading them into memory naively. While you may not notice this on small files but as soon as you have to deal with large files loading them into memory at once becomes quite costly.

Most .NET methods that work with byte[] already exhibit Stream counterpart so it shouldn’t be a problem to use it. When you provide your own API, you should consider supplying a method that operates with Stream in a robust batch-by-batch fashion.

Comparing streams

Imagine we want to write utility method that compares two streams. We could utilize the approach mentioned above: read chunks from the stream and compare these chunks. However, there are some dangers hidden in this approach. For some implementations of Stream such as NetworkStream the number of actual bytes read may differ from the expected number of bytes.

To combat this we’ll compare streams byte-by-byte.

public static bool IsEqual(this Stream stream, Stream otherStream)
    if (stream is null || stream.Length == 0) return false;

    if (otherStream is null || otherStream.Length == 0) return false;

    if (stream.Length != otherStream.Length) return false;

    int buffer;
    int otherBuffer;

    while (true)
        buffer = stream.ReadByte();
        if (buffer == -1) break;
        otherBuffer = otherStream.ReadByte();

        if (buffer != otherBuffer)
            stream.Seek(0, SeekOrigin.Begin);
            otherStream.Seek(0, SeekOrigin.Begin);
            return false;

    stream.Seek(0, SeekOrigin.Begin);
    otherStream.Seek(0, SeekOrigin.Begin);

    return true;


Stream APIs allow batch-by-batch processing which allows us to reduce memory consumption on big files. At the first glance, Stream API may seem as requiring more ceremony, it’s definitely a useful tool in one’s toolbox.


  • 25th July, 2021 - Published initial version
  • 9th August, 2021 - Updated stream comparison code according to comments
  • 13th July, 2022 - Added reference source for more clarity
  • 12th April, 2023 - Fixed stream comparison for network streams


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Written By
Team Leader
Ukraine Ukraine
Team leader with 8 years of experience in the industry. Applying interest to a various range of topics such as .NET, Go, Typescript and software architecture.

Comments and Discussions

QuestionPerformance Pin
honey the codewitch12-Apr-23 23:41
mvahoney the codewitch12-Apr-23 23:41 
AnswerRe: Performance Pin
Graeme_Grant13-Apr-23 2:53
mvaGraeme_Grant13-Apr-23 2:53 
GeneralRe: Performance Pin
honey the codewitch13-Apr-23 4:33
mvahoney the codewitch13-Apr-23 4:33 
GeneralRe: Performance Pin
Graeme_Grant13-Apr-23 7:45
mvaGraeme_Grant13-Apr-23 7:45 
GeneralRe: Performance Pin
honey the codewitch13-Apr-23 9:28
mvahoney the codewitch13-Apr-23 9:28 
SuggestionRe: Performance Pin
Chad3F18-Apr-23 17:49
Chad3F18-Apr-23 17:49 
AnswerRe: Performance Pin
Bohdan Stupak19-Apr-23 4:46
professionalBohdan Stupak19-Apr-23 4:46 
GeneralRe: Performance Pin
honey the codewitch19-Apr-23 4:56
mvahoney the codewitch19-Apr-23 4:56 
QuestionWhy 2KB and not 4KB (page alignment)? Pin
James McC.9-Oct-22 12:10
professionalJames McC.9-Oct-22 12:10 
AnswerRe: Why 2KB and not 4KB (page alignment)? Pin
Bohdan Stupak12-Apr-23 6:15
professionalBohdan Stupak12-Apr-23 6:15 
QuestionA Comparison with a NuGet package Pin
George Swan19-Jul-22 22:14
mveGeorge Swan19-Jul-22 22:14 
AnswerRe: A Comparison with a NuGet package Pin
Bohdan Stupak26-Jul-22 23:42
professionalBohdan Stupak26-Jul-22 23:42 
QuestionMeasurements Pin
Mobster_11-Aug-21 13:50
Mobster_11-Aug-21 13:50 
AnswerRe: Measurements Pin
Bohdan Stupak26-Sep-21 3:51
professionalBohdan Stupak26-Sep-21 3:51 
GeneralRe: Measurements Pin
Mobster_26-Sep-21 5:58
Mobster_26-Sep-21 5:58 
QuestionOne more advantage of Stream over byte[] is also on async situations Pin
Adérito Silva30-Jul-21 6:26
Adérito Silva30-Jul-21 6:26 
Your article is very good, it demonstrates several advantages of Stream over byte arrays. For people reading this, I would also like to mention the situations where we are reading data from files or network and the advantages Stream provides over byte[] in these contexts.

For example, Stream offers ReadAsync and WriteAsync methods, which may use internal I/O to read and write data asynchronously without using additional threads (depending on the derived class's implementation). When compared to some static methods like System.IO.File.ReadAllBytes or System.IO.File.WriteAllBytes or similar blocking methods, this poses considerable advantages, specially when the main application thread needs to be free and responsive (like applications with UI). Many times, blocking methods that allow to read to or write from a byte array, also provide an async variant, but using them allocates byte arrays anyway, which disadvantages are very well demonstrated by this article.

Usually, asynchronous code itself requires an experienced developer, who would already know very well when to use Stream and when to use byte[] efficiently, but less experienced developers should benefit from knowing this is one of the advantages of Stream, when using its async methods.

This article is important, because it shows how a simple bad habit like the overuse of byte arrays could negatively affect performance. Many thanks to the author for taking the time to measure this and writing an article that compares several situations! Thumbs Up | :thumbsup:
AnswerRe: One more advantage of Stream over byte[] is also on async situations Pin
Bohdan Stupak9-Aug-21 2:56
professionalBohdan Stupak9-Aug-21 2:56 
QuestionChecking two streams, a couple of suggestions. Pin
George Swan27-Jul-21 20:28
mveGeorge Swan27-Jul-21 20:28 
AnswerRe: Checking two streams, a couple of suggestions. Pin
Bohdan Stupak30-Jul-21 3:42
professionalBohdan Stupak30-Jul-21 3:42 
AnswerRe: Checking two streams, a couple of suggestions. Pin
Bohdan Stupak9-Aug-21 2:59
professionalBohdan Stupak9-Aug-21 2:59 
GeneralShould use array only when array size is known at design time. Pin
Adérito Silva25-Jul-21 6:22
Adérito Silva25-Jul-21 6:22 
GeneralRe: Should use array only when array size is known at design time. Pin
Bohdan Stupak30-Jul-21 3:41
professionalBohdan Stupak30-Jul-21 3:41 
QuestionQuestion about the definition of "equal" Pin
Nelek25-Jul-21 0:56
protectorNelek25-Jul-21 0:56 
AnswerRe: Question about the definition of "equal" Pin
Bohdan Stupak25-Jul-21 1:17
professionalBohdan Stupak25-Jul-21 1:17 
GeneralRe: Question about the definition of "equal" Pin
Nelek25-Jul-21 8:44
protectorNelek25-Jul-21 8:44 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.