appending and splitting files larger than 2gb

Yes it can be done - I believe, I haven't tried it (I don't have many 2Gb files lying around, and those I do I don't particularly want to but together)

The problem is that you are trying to do it in a single chunk, via File.ReadAllBytes, which is (understandably) limited to 2Gb.

If you read it in chunks of say 100Mb then that will be a lot easier for the system to handle and should work fine.

See http://msdn.microsoft.com/en-us/library/system.io.filestream.read.aspx[^] and you'll be fine.

You have to read the file you want to append to the other in smaller chunks inside a loop. There is a limit of 2GB for one process (at least on 32bit machines). So if the function were to read the whole 2GB at once the process would run out of memory.

Besides that: fSource is a bit misleading here. It should be more like fDest since that file handle is the destination or the write operation. fSource should be the file Part2FilePath. Open that and read it in chunks of 100 - 500MB which you then append to fDest.

Best Regards,
Manfred

Is it for your personal use or is it for an application that you are developing? If it is for an in-house purpose, you might save time using System.Diagnostics.Process.Start() to call the tool that you downloaded. If the tool doesn't support command line, you could use some other free tools. 7-zip[^] is primarily a free compression software, but you can use it with a compression level of 0 to join files. Uncompressing will split the files.

Read this article. Hope it will benefit you
Split large XML files into small files[^]

Good Luck :)

Thank you everyone for your help. I always appreciate it and every answer here was a good one. I gave everyone a 5.

I am using an ftp api that downloads a 5g file everynight. The api requires on fail to make a new file then merge them. This is why i had that problem. I agree its not the best way but I have to use it per company policy.

I used OriginalGriff's answer. I ran a for loop broke it into 20 pieces and then put it back together with the original file.

Here is the code i used. I tried it on a short text file, zip file and exe. It worked on all three. I tried to format the code here I hope it looks ok. Thanks again.

Brian Cummings

for (int x = 0; x < 20; x++)
{
  try
  {
    using (FileStream fsSource = new FileStream(pathSource,
                        FileMode.Open, FileAccess.Read))
    {
    // Read the source file into a byte array.
    long fLength = fsSource.Length / 20 + 1;
    byte[] bytes = new byte[fLength];
    int numBytesToRead = (int)fLength;
    int numBytesRead = 0;
    while (numBytesToRead > 0)
    {
       fsSource.Position = readPosition;
       readPosition += fLength;
       int n = fsSource.Read(bytes, numBytesRead, numBytesToRead);
       
       // Break when the end of the file is reached.
       if (n == 0) break;
       
       numBytesRead += n;
       numBytesToRead -= n;
    }
    
   numBytesToRead = bytes.Length;
   // Write the byte array to the other FileStream.
   using (FileStream fsNew = new FileStream(pathNew + "00" +   x.ToString() + ".zip", FileMode.Create, FileAccess.Write))
   {
      fsNew.Write(bytes, 0, numBytesToRead);
      fsNew.Flush();
      fsNew.Close();
      fsNew.Dispose();
    }//using
 }//using
}//try
catch (Exception ex)
{
   MessageBox.Show(ex.Message);
   Console.WriteLine(ex.Message);
}

}//for loop x