Click here to Skip to main content
15,889,877 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I',m trying to figure out how to do a multipart/form-data put request and be able to get upload the progress of my file. The problem I am having is that the request is not sent until request.GetResponse(); making my writing progress go to 100% before sending. I have tried googling and found people solving it by using request.SendChunked = true; and request.AllowWriteStreamBuffering = false; but this does not seem to do any difference. I have seen several people post about this but not found a solution that I get working, yet. This is my attempt. Any idea what I can do to solve this?

What I have tried:

<pre lang="c#">
class Program
    {
        private static float _progressBar;

        static async Task Main(string[] args)
        {
            Console.WriteLine("hello");
            // https://postman-echo.com/put
            var response =  ExecutePutCommand("C:/temp/zipfile.zip",
                "https://postman-echo.com/put", "weef");
         }

        protected static string ExecutePutCommand(string filename, string uriStr, string stringToAdd)
        {
            try
            {
                const string authCookie = "";
                byte[] fileBytes = File.ReadAllBytes(filename);
                //
                HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uriStr);
                request.SendChunked = true;
                request.PreAuthenticate = true;
                request.AllowWriteStreamBuffering = false;
                request.KeepAlive = true;
                request.ProtocolVersion = HttpVersion.Version11;

                string boundary = System.Guid.NewGuid().ToString();
                //
                request.ContentType = string.Format("multipart/form-data;boundary={0}", boundary);
                request.Method = "PUT";
                //request.Headers["Cookie"] = authCookie;

                // Build Contents for Post
                string header = string.Format("--{0}", boundary);
                string footer = header + "--";
                // string builders are for the text above and below the file - file is kept in its original format.
                StringBuilder contents = new StringBuilder();
                StringBuilder contents2 = new StringBuilder();

                // Zip File
                contents.AppendLine(header);
                contents.AppendLine(
                    string.Format("Content-Disposition:form-data; name=\"filefield\"; filename=\"{0}\"",
                        filename));
                contents.AppendLine("Content-Type: application/x-zip-compressed");
                contents.AppendLine();

                // file goes here... then contents 2

                contents2.AppendLine();
                // Form Field 1
                contents2.AppendLine(header);
                contents2.AppendLine("Content-Disposition:form-data; name=\"stringToAdd\"");
                contents2.AppendLine();
                contents2.AppendLine(stringToAdd);
                // Form Field 2
                // Footer
                contents2.AppendLine(footer);

                // This is sent to the Put
                byte[] bytes = Encoding.UTF8.GetBytes(contents.ToString());
                byte[] bytes2 = Encoding.UTF8.GetBytes(contents2.ToString());
                byte[] formData = bytes.Concat(fileBytes).Concat(bytes2).ToArray();
               
                // calculate size of the stream
                request.ContentLength = bytes.Length + fileBytes.Length + bytes2.Length;

                byte[] buffer = new byte[4096];
                int count = 0;
                int length = 0;
                Stopwatch sw = new Stopwatch();
                sw.Start();
                //request.Proxy = null;
                using (Stream requestStream = request.GetRequestStream())
                {
                    using (Stream inputStream = new MemoryStream(formData))
                    {

                        while ((count = inputStream.Read(buffer, 0, buffer.Length)) > 0)
                        {
                            requestStream.Write(buffer, 0, count);
                            length += count;

                            _progressBar = (float)length / request.ContentLength;
                            //Console.WriteLine(_progressBar);
                        }
                    }
                }
                sw.Stop();
               
                WebResponse resp =  request.GetResponse();
                Stopwatch sw2 = new Stopwatch();
                sw.Start();
                using (Stream stream = resp.GetResponseStream())
                {
                    StreamReader respReader = new StreamReader(stream);
                    sw2.Stop();
                    return respReader.ReadToEnd();
                }
            }
            catch (Exception e)
            {

                Debug.WriteLine(e.ToString());
                return String.Empty;
            }

            return null;
        }
    }




If i do a curl from the command line it will start a tickingupload progress text that shows how many mb has been uploaded and expected time left so the postman server seems to support this functionality:


C:\temp>curl -v -X PUT -F "filefield=@C:\temp\zipfile.zip" -F "field2=blah" https://postman-echo.com/put > log.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 107.23.124.180...
* TCP_NODELAY set
* Connected to postman-echo.com (107.23.124.180) port 443 (#0)
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 181 bytes...
* schannel: sent initial handshake data: sent 181 bytes
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 2/3)
* schannel: encrypted data got 4096
* schannel: encrypted data buffer: offset 4096 length 4096
* schannel: encrypted data length: 4030
* schannel: encrypted data buffer: offset 4030 length 4096
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 5054 length 5054
* schannel: encrypted data length: 200
* schannel: encrypted data buffer: offset 200 length 5054
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 2/3)
* schannel: encrypted data got 147
* schannel: encrypted data buffer: offset 347 length 5054
* schannel: sending next handshake data: sending 126 bytes...
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 2/3)
* schannel: encrypted data got 258
* schannel: encrypted data buffer: offset 258 length 5054
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with postman-echo.com port 443 (step 3/3)
* schannel: stored credential handle in session cache
> PUT /put HTTP/1.1
> Host: postman-echo.com
> User-Agent: curl/7.55.1
> Accept: */*
> Content-Length: 11472946
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------3afff00f52d07df7
>
* schannel: client wants to read 102400 bytes
* schannel: encdata_buffer resized 103424
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: encrypted data got 54
* schannel: encrypted data buffer: offset 54 length 103424
* schannel: decrypted data length: 25
* schannel: decrypted data added: 25
* schannel: decrypted data cached: offset 25 length 102400
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: decrypted data buffer: offset 25 length 102400
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 25
* schannel: decrypted data buffer: offset 0 length 102400
< HTTP/1.1 100 Continue
} [160 bytes data]


 **31 10.9M    0     0   31 3552k      0   208k  0:00:53  0:00:17  0:00:36  317k**
Posted
Updated 21-Dec-20 11:09am

1 solution

Everybody fakes progress until they're sure what it is. The point is amusing the user, not accuracy.

You run some tests, get an average, and guesstimate based on that, revising as you get feedback.

"Moving parts" (versus a frozen progress bar) is the key.
 
Share this answer
 
Comments
endasil 21-Dec-20 17:31pm    
No that is simply incorrect. While the time to finish is an estimate the amount of bytes transferred is an exact number. If you know how big the file is and how many bytes you have sent, you can show how many percentages of the file there are left to upload. The question is how to perform a continuous chunked transfer instead of buffering everything and then transferring. Use curl and you will see how much of the file has been uploaded.
Dave Kreskowiak 21-Dec-20 18:49pm    
Welp, you have no choice but to fake it. HttpWebRequest does not report progress so there's no way for you to know how much of the file has been uploaded.

To get any kind of progress, you would have to write your own version of HttpWebRequest and implement reporting progress.
endasil 22-Dec-20 3:31am    
I think a better description of what I want to do is to send a partial request, for example, send 4000k, then send 4000k more. To have the request stream send the chunks of data I write to it. To have the request stream actually send the data when i write to it.I would have guessed that since HttpWebRequest has support for chunked encoding there would be some way of sending data in chunks. Someone has to have a need for this before me, it's not like it is a totally new concept.

Do you have any suggestions on other methods to use for solving this problem? Otherwise, what I could read up on regarding making my own version of HttpWebRequest?
Dave Kreskowiak 22-Dec-20 10:01am    
What you're missing is the server must ALSO support chunked data.

You would have to write your server-side code to accept data in chunks and now how to stitch the chunks back together from multiple requests. You can only send one chunk at a time and each request will not know the state of the previous requests so your server-side code has to keep track of that somehow and know when to attempt to stich the data back together.

Your client-side code will also have to read the data it's sending, chunk by chunk, making a POST or PUT request for each chunk, also with data telling the server what chunk this is, the order number of the chunk, any error checking, ...

Oh, and your client-side code does NOT have direct access to the client file system, so it's cannot directly open files for reading. This is because its considered a security risk, for very good reasons.
endasil 22-Dec-20 13:29pm    
The server supports this since it works with curl.

If you read my code you can see that I read the file and store it in memory. I have stored the whole request in the variable buffer and then read it with count = inputStream.Read(buffer, 0, buffer.Length). Then write it to the requestStream with requestStream.Write(buffer, 0, count);

There is a HttpWebRequest.SendChunked Property so apparently there is some handling of sending chunks of data built-in, it feels silly to not have any control of it.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900