Read Large Files Into Memory - C# Discussion Boards

23-May-07 20:54

I am trying to decompress a file. To do this, with my current code which you can see below, I am reading the entire file into a byte[]. But if my file is big, I believe bigger then 64 KB, then I can't declare a byte[] big enough to hold the file. How do I fix this, work around this?

---------Code---------
public void GZipDeCompressStream(Stream InStream, Stream OutStream)
{
//Decompresser
GZipStream gzDecompressed = new GZipStream(InStream, CompressionMode.Decompress, true);

//Retrieve the size of the decompressed file from the compressed footer
byte[] bufferWrite = new byte[4];
InStream.Position = (int)InStream.Length - 4;
InStream.Read(bufferWrite, 0, 4);
InStream.Position = 0;

//Convert to int for using in declaring our Byte[] size
int bufferLength = BitConverter.ToInt32(bufferWrite, 0);
//Create our Buffer: size + 100
------------//This is where my issue is. Buffer Lenght is WAY bigger then the 64 KB limit.
byte[] buffer = new byte[bufferLength + 100];
int readOffset = 0;
int totalBytes = 0;

// Loop through the compressed stream and put it into the buffer
while (true)
{
int bytesRead = gzDecompressed.Read(buffer, readOffset, 100);

// If we reached the end of the data
if (bytesRead == 0)
break;

readOffset += bytesRead;
totalBytes += bytesRead;
}

// Write the content of the buffer to the destination stream
OutStream.Write(buffer, 0, totalBytes);

// Close the streams
InStream.Close();
gzDecompressed.Close();
OutStream.Close();
}

Christian Graus23-May-07 21:21

23-May-07 21:21

is long bigger than int in C# ? I thought it was.

Christian Graus - Microsoft MVP - C++
Metal Musings - Rex and my new metal blog
"I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )

Patricker23-May-07 21:29

23-May-07 21:29

This post does not make sense in the context of my question, or you didn't explain yourself well enough.

Christian Graus23-May-07 21:57

23-May-07 21:57

You're storing the size in an int. A long is bigger.

Patricker23-May-07 22:00

23-May-07 22:00

Ahh. The problem wasn't int being big enough. The problem was making a byte[] that was big enough. My byte[]'s total size in memory was greater then 64kb. The other post though should solve my issues.

DavidNohejl24-May-07 0:27

DavidNohejl

24-May-07 0:27

Yeah, long is System.Int64, and int is System.Int32;

"Throughout human history, we have been dependent on machines to survive. Fate, it seems, is not without a sense of irony. " - Morpheus
"Real men use mspaint for writing code and notepad for designing graphics." - Anna-Jayne Metcalfe

lmoelleb23-May-07 21:49

lmoelleb

23-May-07 21:49

There is no 64KB limit on arrays. If you get close to 2GB you might have problems on 32 bit systems.

But anyway, the solution to your problem is quite simple - no need to keep all the data in memory, just save it as you go:

byte[] buffer = new byte[1024*1024]; // 1MB buffer
while (true)
{
int bytesRead = gzDecompressed.Read(buffer, 0, buffer.Length);

// If we reached the end of the data
if (bytesRead == 0) break;
outStream.Write(buffer, 0, bytesRead)
}

Patricker23-May-07 23:22

2 questions regarding graphics

23-May-07 23:22

You are awesome. That totally solved my problem and she works great now.

Thanks a Million.

--Peter

Muammar©23-May-07 20:43

23-May-07 20:43

Hi,
1. How can you resize a picture stored in a Bitmap object.
2. Why is it not a real icon when you save it as ImageFormat.Icon

Thanks Smile | :)

Smile: A curve that can set a lot of things straight!
(\ /)
(O.o)
(><)

Christian Graus23-May-07 20:47

23-May-07 20:47

1 - You can create a new bitmap of the right size, and copy it through a graphics object. You can also just display it at a different size, if that's all you need.
2 - I have no idea, what makes it not real ?

Muammar©23-May-07 20:56

23-May-07 20:56

Thank you Christian, regarding (2), simply when you save it as an icon that way, it's not displayed along with the other icons as "an icon", you dont see the icon!! you just see the file as if it's a corrupted file and the only way to view it is to set the container folder view as "thumbnail", simply it still acts like a picture file with a missing icon!

Smile: A curve that can set a lot of things straight!
(\ /)
(O.o)
(><)

Guffa23-May-07 21:04

Guffa

23-May-07 21:04

Muammar© wrote:
How can you resize a picture stored in a Bitmap object.

Create a new Bitmap object with the size you want, create a Graphics object for it, and draw the original image on it scaled to the size.

Muammar© wrote:
Why is it not a real icon when you save it as ImageFormat.Icon

In what way is it not a "real" icon?

Have you included any mask in the image? I don't know how to do this exactly, but I suppose that you would add it as another frame to the image.

---
single minded; short sighted; long gone;

23-May-07 21:09

mask.. frame.. Even more puzzled!

Smile: A curve that can set a lot of things straight!
(\ /)
(O.o)
(><)

Christian Graus23-May-07 21:19

23-May-07 21:19

Icons use a specific color as a transparency mask.

23-May-07 21:38

bm.MakeTransparent(colorDialog1.Color); still not working.. Please christian, help Frown | :(

Smile: A curve that can set a lot of things straight!
(\ /)
(O.o)
(><)

Martin#23-May-07 21:43

Martin#

23-May-07 21:43

Hello,

You can get the Color directly from your Bitmap:

Color transparentColor = bm.GetPixel(0, 0);
bm.MakeTransparent(transparentColor);

Hope it helps!

All the best,

MArtin

23-May-07 21:55

Thanks Martin, but I tried this as well.. nope, still not working, are you telling me that it's working fine with everyone here??

Smile: A curve that can set a lot of things straight!
(\ /)
(O.o)
(><)