Click here to Skip to main content
15,888,610 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi,

I have a load of images stored in zip files. I am trying to return the image from my Web Api 2 application. Can anyone suggest a clever way to return the stream directly from the zip and then dispose of it (code below)?

I'm considering scrapping the 'zip' plan and storing the images as flat files, but there are literally millions of them. I worry about how much space they will take on disk.

I am looking for the fastest possible way to deliver the image from the time of request. That is my main concern for this PoC.

Thanks ^_^
Andy

What I have tried:

Currently I am copying the stream to a memory stream:

I store the path as "C:\ImageLib\1045.zip\GS67801_POS_Image.jpg"
C#
private Stream GetImageByItemCode(string itemCode)
        {
            var item = ItemType.SelectByItemCode(itemCode).FirstOrDefault();
            if (item == null)
                return null;

            if (item.PosImagePath == null)
                return null;
            MemoryStream ms = new MemoryStream();
            if (item.PosImagePath.IndexOf(".zip") > 0)
            {
                var zipPath = item.PosImagePath.Substring(0, item.PosImagePath.IndexOf(".zip") + 4);
                var filename = item.PosImagePath.Substring(item.PosImagePath.IndexOf(".zip") + 5);
                var zipFile = ZipFile.Open(zipPath, ZipArchiveMode.Read);

                var entry = zipFile.Entries.Where(e => e.FullName == filename).FirstOrDefault();
                if (entry == null)
                    return null;

                entry.Open().CopyTo(ms);

                ms.Seek(0, SeekOrigin.Begin);
            }
            return ms;
        }

This is a PoC so I haven't optimized how I store the filename atm. That's just detail I can deal with later.

This method is used to create an HttpResponse:
C#
public HttpResponseMessage GetImageImg(string itemCode, int size)
{
    Stream imageStream = GetImageByItemCode(itemCode);

    //var imageArray = ResizeImage(image, size);



    var response = Request.CreateResponse();
    if (imageStream == null)
    {
        return response;
    }
    response.Content = new StreamContent(imageStream);
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
    return response;
}

which is returned by my controller here:

C#
[Route("{itemCode}/{size:int?}")]
   public object GetImage(string itemCode, int size = 1)
   {
       var query = Request.GetQueryNameValuePairs().ToDictionary(p => p.Key.ToLower(), p => p.Value.ToLower());
       var format = "img";
       if (query.ContainsKey("format"))
       {
           format = query["format"];
       }
       switch (format)
       {
           default:
               return GetImageImg(itemCode, size);
       }
   }

Don't worry about all that formatting crap. Eventually the size / format will be stored in my lib as separate files.
Posted
Updated 25-Sep-17 10:23am

1 solution

JPG is a compressed format already.

How much are these images being compressed in the .ZIP files? Most image formats, like PNG, JPG, TIFF, ... are already compressed so you're not really getting much by putting them into .ZIP files. You might be getting 1% to 2% compression.

Or, if you're using a non-compressed format, like BMP, can you get away with converting and using a compressed version? Some formats and settings will be lossy and others won't. It depends on the images.

As for the stream operation on the file, that depends on the ZIP library you're using. If it's the one in the System.IO.Compression namespace built into the .NET Framework, you can get the entry for the file using the ZipArchive.GetEntry Method (String) (System.IO.Compression)[^] method to get a ZipArchiveEntry object and then use the ZipArchiveEntry.Open Method (System.IO.Compression)[^] method to get the stream.
 
Share this answer
 
Comments
Andy Lanng 26-Sep-17 3:42am    
A: you didn't read my current solution
B: your answer offers me no more information than I gave
C: jpeg has compression levels. You can easily achieve much higher in a zip. Also, each zip file contains hundreds of images to better ratios are likely. Try 80% compression ratio
Sorry to sound harsh but you really didn't answer my question of how to return the stream from ZipArchiveEntry.open() before disposing of the archive
Dave Kreskowiak 26-Sep-17 7:56am    
Sorry, I missed the code.

I fully realize JPG has compression levels. Most are compressed to the point where a ZIP is pointless. If you've got JPG's that are not compressed as much, lucky you.

You're already returning the stream as fast as possible. Since the original ZIP is closed and disposed at the time, the content has to be transferred to another stream that will live long enough.

You're only other option to return the content faster is to have the files not in a ZIP but on disk and just have the web server return the file directly by URL.

If space is an issue, try converting the images to a different format and compression. Yes, I know you have "millions" of them.
Andy Lanng 26-Sep-17 10:53am    
Yeah - That's kinda the conclusion I got to. The images are supposed to be high-res (dunno why they chose jpg!). Space doesn't have to be an issue. Just adds to the cost.

Thanks for getting back to me ^_^
Dave Kreskowiak 26-Sep-17 12:14pm    
I work on a site where we allow users to upload images to be used in documentation. These are normally screenshots, but can be anything.

We migrated from an old (piece of sh*t!) site to one we rewrote from scratch. The old site had about 780GB of images, of all types like JPG, BMP, some TIFF, ... We had tons of BMP's and they are the worst as they are 32bpp and no compression.

After testing on some sets of these images, I found that, in our case, PNG works the best for compressing the images down and without loosing image quality. YOUR MILEAGE MAY VARY.

Converting all of our images to PNG got that 780GB down to 2GB, with 80x80 thumbnails along side them! We have a controller and library that converts any image uploaded in any format to PNG automatically before being shipped off to the database.

For good reasons I can't go into, we keep the images stored in the database, not in the server file system. It's not the most performant way to store the images, but it's what we require.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900