Click here to Skip to main content
15,891,136 members
Please Sign up or sign in to vote.
1.00/5 (2 votes)
See more:
how can i chunk files and load them into memory and then into db in small steps like 64mb ?

is there any code for that ?
Posted
Comments
ZurdoDev 8-Jan-16 7:57am    
There is a lot of code for it, more than we can just give you here. Where are you stuck?
Andy Lanng 8-Jan-16 8:05am    
This story has a bit of history. Check out the OPs previous post
ZurdoDev 8-Jan-16 8:18am    
Suppose you should give OP your personal email and start tutoring then. ;)
Andy Lanng 8-Jan-16 8:22am    
O_o
Email? what's that?
*hides behind desk*

Hi again :D
You can read a file into a byte buffer one chunk at a time, or you can split an existing byte[] into several chunks. It's pretty common practice so there's lots on google to help

This link should be helpful for reading data in byte[] chunks

in particular this example given in the second link writes the "chucks" to a memory stream. You can write these chunks anywhere:
C#
/// <summary>
/// Reads data from a stream until the end is reached. The
/// data is returned as a byte array. An IOException is
/// thrown if any of the underlying IO calls fail.
/// </summary>
/// <param name="stream">The stream to read data from</param>
public static byte[] ReadFully (Stream stream)
{
    byte[] buffer = new byte[32768]; //set the size of your buffer (chunk)
    using (MemoryStream ms = new MemoryStream()) //You need a db connection instead
    {
        while (true) //loop to the end of the file
        {
            int read = stream.Read (buffer, 0, buffer.Length); //read each chunk
            if (read <= 0) //check for end of file
                return ms.ToArray();
            ms.Write (buffer, 0, read); //write chunk to [wherever]
        }
    }
}



I hope that helps.
PS: Just curious, what are you writing? trying to hide files? there are better ways.
 
Share this answer
 
Comments
brandon1999 8-Jan-16 8:16am    
hi :D

i'm making a program that save large files and decrease the size and when u click on some files it create that file in C/temp and u can use that file and when u close the program it will delete that temp folder automaticlly.

thank u so much that was helpful.
Open a stream, and use teh Read[^] method to fetch a chunk of data. That's the simple bit.
Adding it to the DB is the difficult part: unless you are reading text from the file, you can't "append" the data into a DB column (and even with text, that would be monumentally inefficient), so you would have to assemble all the chunks into a single "block" to send it to the DB anyway. And if you are doing that, you might as well just use File.ReadAllBytes to get the lot and pass that to the DB in the first place.
The only time when this is really silly is when your file sizes start to get huge - and if you are storing huge files in your DB then that's probably a major mistake anyway!
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900