Introduction
Hello, this is my first article on CodeProject. I have been a long time
reader, and the CodeProject resource has been an endless supply of answers to
many questions. After searching CodeProject, I found that the .NET section
lacked any articles on compression, so I thought I would write this article.
SharpZipLib from ICSharpCode
First of all, this article depends on the SharpZipLib which is 100% free to
use, in any sort of projects. Details on the license and download links are
available here.
Purpose
A friend asked me to teach him C#.NET, and as a project to teach him, I
decided to start writing a revision control system utilizing both server and
client, we've both had our share of pitfalls with CVS. One of the features he
wanted involved compression, so I sought out this library, but its documentation
is sketchy unless you use it purely for an API reference. Also, the
documentation only shows examples of file based compression. However, in our
project, we wanted the ability to work in memory (with custom diff-type
patches). Originally, I found this library on a forum that said this wasn't
possible, but after digging into the library documentation, I found some
Stream-oriented classes that looked promising. An hour or so of playing around,
and this simple and short code was the result. Since the code is relatively
short, I have not included any source or demo files to download. I hope someone
finds this useful!
Compression
For convenience sake, we localize the namespaces IO
,
Text
, and SharpZipLib
:
using System;
using System.IO;
using System.Text;
using ICSharpCode.SharpZipLib.BZip2;
First of all, we'll start with compression. Since we're using
MemoryStream
s, let's create a new one:
MemoryStream msCompressed = new MemoryStream();
Simple enough, right? For this example, I will use BZip2. You can use Zip, or
Tar, however, they require implementing a dummy FileEntry
, which is
extra overhead that is not needed. My choice of BZip2 over GZip comes from the
experience that larger data can be compressed smaller, at the cost of a slightly
larger header (discussed below).
Next, we create a BZip2 output stream, passing in our
MemoryStream
.
BZip2OutputStream zosCompressed = new BZip2OutputStream(msCompressed);
Pretty easy... Now however, is a good time to address the header overhead I
mentioned above. In my practical tests, compressing a 1 byte string, rendered a
28 byte overhead from the headers alone when using GZip, plus the additional
byte that could not be compressed any further. The same test with BZip2 rendered
a 36 byte overhead from the headers alone. In practice, compressing a source
file from a test project of 12892 bytes was compressed to 2563 bytes, about a
75% compression rate give or take my bad math, using BZip2. Similarly, another
test revealed 730 bytes compressed to 429 bytes. And a final test, a 174 bytes
compressed to 161 bytes.
Obviously, with any compression, the more data is available, the better the
algorithm can compress patterns.
So with that little bit of theory out of the way, back to the code... From
here, we start writing data to the BZip2OutputStream
:
string sBuffer = "This represents some data being compressed.";
byte[] bytesBuffer = Encoding.ASCII.GetBytes(sBuffer);
zosCompressed.Write(bytesBuffer, 0, bytesBuffer.Length);
zosCompressed.Finalize();
zosCompressed.Close();
Pretty easy. As with most IO and stream methods, byte arrays are used instead
of strings. So we encode our output as a byte array, then write it to the
compression stream, which in turn compresses the data and writes it to the inner
stream, which is our MemoryStream
.
bytesBuffer = msCompressed.ToArray();
string sCompressed = Encoding.ASCII.GetString(bytesBuffer);
So now, the MemoryStream
contains the compressed data, so we
pull it out as a byte array and convert it back to a string. Note that this
string is NOT readable, attempting to put this string into a textbox will render
strange results. If you want to view the data, the way I did it was to convert
it into a Base64 string, but this increases the size, anyone has any suggestions
to that are welcome to comment. The result of running this specific code renders
the 43 byte uncompressed data as 74 byte compressed data, and when encoded as a
base 64 string, the final result is 100 characters as follows:
QlpoOTFBWSZTWZxkIpsAAAMTgEABBAA+49wAIAAxTTIxMTEImJhNNDIbvQ
aWyYEHiwN49LdoKNqKN2C9ZUG5+LuSKcKEhOMhFNg=
Obviously, these are not desirable results. However, I believe the speed of
which the library compresses short strings of data could be extended into a
method which returns either a compressed or uncompressed string with a flag
indicating which was more efficient.
Uncompression
Now of course, to test our code above, we need some uncompression code. I
will put all the code together, since it's pretty much the same, just using a
BZip2InputStream
instead of a BZip2OutputStream
, and
Read
instead of Write
:
MemoryStream msUncompressed =
new MemoryStream(Encoding.ASCII.GetBytes(sCompressed));
BZip2InputStream zisUncompressed = new BZip2InputStream(msUncompressed);
bytesBuffer = new byte[zisUncompressed.Length];
zisUncompressed.Read(bytesBuffer, 0, bytesBuffer.Length);
zisUncompressed.Close();
msUncompressed.Close();
string sUncompressed = Encoding.ASCII.GetString(bytesBuffer);
Now, a quick check on sUncompressed
should reveal the original
string intact... No files involved, however, if you wanted to load a file, there
are a few ways you can do it, and I leave it to your imagination.
Closing
Special thanks to the developers at ICSharpCode.Net for providing this
awesome library free to the public which makes this article possible. I have
no affiliation with ICSharpCode.Net, so I hope I have not breached anything in
posting this article.
I hope you all find this as useful as I have!
Short and simple, I'm a self contracted programmer, my strongest programming skills are in C/C++ and C#/.NET. I have a nack for porting C algorithms to C#.