.net C# downloading files using system.net.webclient

Question

0.00/5 (No votes)

See more:

I have gotten a weird issue. The code I am using was actually used before and it actually worked. Now, I am unsure what has happened, but now the code is not working. It will place a file and will not add its content at all. The url/uri is correct, because I checked by using breakpoints and manually entering that url in a webbrowser. I am using .NET 5.0.

What I have tried:

C#

public void BeginDownload(string url, string destination)
{
    Uri uri = new Uri(url);
    string filename = url.Substring(url.LastIndexOf("/") + 1);
    webClient = new WebClient();
    webClient.DownloadFileCompleted += WebClient_DownloadFileCompleted;
    webClient.DownloadProgressChanged += WebClient_DownloadProgressChanged;
    String newFile = destination + filename;
    webClient.DownloadFileAsync(uri, @newFile);
}

Posted 13-Feb-22 13:28pm

CoderzF1

Updated 14-Feb-22 13:39pm

Add a Solution

2 solutions

Solution 1

Your code would fail if destination doesn't end on a backslash.
Rather than string concatenations, you'd better use appropriate methods to parse or build paths. such as Path.GetFileName() and Path.Combine().

:)
I

Posted 13-Feb-22 14:04pm

Luc Pattyn

Updated 13-Feb-22 14:06pm

v2

Comments

CoderzF1 13-Feb-22 20:21pm

seems weird that there is an empty file actually placed in the destination directory. there shouldnt even be a file present at all if the destination is incorrect

Luc Pattyn 13-Feb-22 20:50pm

true. Is the empty file timestamped according to your latest try run?

You did not show the code for Progress and Completed handlers.
If anything goes wrong there, the download might come to a standstill right away.

Luc Pattyn 13-Feb-22 21:14pm

I was able to replicate.
In my case the download did never take off due to a security issue.
This can be seen in the e.Error field at the Completed handler.
It is all about TLS.
What is needed is executing once a statement like this:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;

the latter works on older .NET versions that don't know about Tls12.

I strongly suggest to always provide code that checks the e.Error field in competion handlers (same for BackgroundWorker and many others).

:)

CoderzF1 14-Feb-22 18:40pm

I have tried this security protocol. I have also checked the error message. All it gives is [The remote server returned an error: (500) Internal Server Error.]
When i try to download the file, it adds a file to the directory and it has a time stamp, but it also shows 0 bytes for the file size.

Luc Pattyn 14-Feb-22 18:57pm

From my observation downloading a file from a secured site (https://... URL): creating an empty file is the first thing it does. When that succeeds it starts trying to contact the site and start the download. Which failed without the TLS12 option, and succeeded with it. The failure was a WebException "request aborted. Could not create SSL/TLS secure channel" which is much more specific than a vague "500. Internal error"

I used .NET FrameWork 4.5 but I can't imagine that matters...

Can you download the file manually i.e. with a browser?
if so, does that need login in?
if so, your code needs to provide credentials

Does the problem also exist with other URLs? Can you show a failing URL?

CoderzF1 14-Feb-22 18:59pm

The site doesnt require authentication at all. I have actually opened a new empty project to test with and it downloaded just fine, even without that security protocol. I even used the same url.

CoderzF1 14-Feb-22 19:00pm

This is the file i am testing with
https://emoji.gg/assets/emoji/4782_Brimstone_Like.png

Luc Pattyn 14-Feb-22 19:06pm

I moved my code to .NET5; it worked well for my test file, even without the TLS12 thingy.

I tried your file, it starts well but eventually it fails with a 500.
I'll investigate.

Luc Pattyn 14-Feb-22 19:21pm

OK, it is not your code that fails, it is the server. Probably on purpose.

I switched to synchronous download, added a try catch, and dumped the response stream. I don't understand all of it, but I got the impression that they just want to prevent non-human downloads. It would be rather easy to "borrow" all their stuff if they didn't protect somehow.

The only suggestion I can make is: download what you (might) need manually, and store it elsewhere, e.g. as embedded resources within your app, or on your own web server, etc.

CoderzF1 14-Feb-22 19:12pm

I have a log file that I have added to my project. This is the error message and stacktrace
[CRITICAL]Feb 14, 2022 - 19:10:29 PM <error> - The remote server returned an error: (500) Internal Server Error.
[CRITICAL]Feb 14, 2022 - 19:10:29 PM <error> - at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.WebClient.GetWebResponse(WebRequest request, IAsyncResult result)
at System.Net.WebClient.GetWebResponseTaskAsync(WebRequest request)
at System.Net.WebClient.DownloadBitsAsync(WebRequest request, Stream writeStream, AsyncOperation asyncOp, Action`3 completionDelegate)

CoderzF1 14-Feb-22 19:13pm

What is driving me crazy is that 2 months ago, the file downloaded just fine, with the same code, and with the same application

Luc Pattyn 14-Feb-22 19:22pm

They got smarter? (see my other reply)

CoderzF1 14-Feb-22 19:23pm

Oh My F...n God......... I had to add a user agent

CoderzF1 14-Feb-22 19:24pm

webClient.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; " +
"Windows NT 5.2; .NET CLR 1.0.3705;)");

Luc Pattyn 14-Feb-22 19:28pm

Yep. That seems to work. They didn't get much smarter after all...

Luc Pattyn 14-Feb-22 19:30pm

webClient.Headers.Add("user-agent", "hi there");

is sufficient D)

CoderzF1 14-Feb-22 19:36pm

Thanks for your help though. It is appreciated. I will post a solution for my question.

Luc Pattyn 14-Feb-22 19:41pm

you're welcome.

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CoderzF1 · Accepted Answer · 2022-02-14T13:39:00

I managed to figure out what I needed to do.
I figured out that I needed to set the user agent for the webclient apparently. Still seems weird that the code worked fine 2 months ago. Oh Well. I would like to thank Luc Pattyn for attempting to help...5 stars.

This is the code that actually worked.

C#

public void BeginDownload(string url, string destination)
{
    Uri uri = new Uri(url);
    string filename = url.Substring(url.LastIndexOf("/") + 1);
    webClient = new WebClient();
    webClient.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; " +
"Windows NT 5.2; .NET CLR 1.0.3705;)");
    webClient.DownloadFileCompleted += WebClient_DownloadFileCompleted;
    webClient.DownloadProgressChanged += WebClient_DownloadProgressChanged;
    String newFile = destination + filename;
    webClient.DownloadFileAsync(uri, @newFile);
}