Click here to Skip to main content
15,884,472 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I am working on an application offline website downloader which downloads the complete website just using the provided URL. Firstly, I am fetching all the URL in any array of the given website and then downloading them using HTTPWebRequest. Things are working fine till here. Now I want to apply progress while downloading website using all the fetched URL but I am unable to find a way to do that. Here is my code. First of all i am calling this method on download button click.

C#
 public void FetchUrl(FetcherArgument arg)
{
    if (stop_)
    {
        return;
    }
    lock (urls_)
    {
        string url = arg.url;
        if (SITE_ROOT == "")
        {
            SITE_ROOT = url;
        }
        if (urls_.Contains(url))
        {
            return;
        }
        //ThreadPool.QueueUserWorkItem(new HttpFetcher().FetchByWebClient, arg);
        ThreadPool.QueueUserWorkItem(new HttpFetcher().Fetch, arg);
        urls_[url] = true;
    }
}


C#
and then call this method

       public void Fetch(Object thread_content_)
   {
        if (FetcherController.Get().Stop())
        {
            return;
        }
        FetcherArgument arg = (FetcherArgument)thread_content_;
        current_ttl_ = arg.ttl;
        index_page_ = arg.index_page;
        string url = arg.url;
        HttpWebRequest request = null;
        HttpWebResponse response = null;
        try
        {
            request = (HttpWebRequest)WebRequest.Create(url);
        }
        catch (NotSupportedException)
        {
            Logger.Log("Url:" + url + " is not supported");
            return;
        }
        try
        {
            response = (HttpWebResponse)request.GetResponse();
        }
        catch (Exception e)
        {
            Logger.Log("Url:" + url + " is not accessible:" + e.ToString());
            return;
        }
        string content_type_primary = arg.content_type_hint;
        if (response.ContentType != null && response.ContentType != "")
        {
            content_type_primary = response.ContentType.ToLower();
            if (content_type_primary.Contains(";"))
            {
      content_type_primary = 
        content_type_primary.Substring(0,content_type_primary.IndexOf(";"));
            }
        }
        Logger.Log("Fetching url:" + url + "[content-type " + content_type_primary + "]");
        switch (content_type_primary)
        {
            case TEXT_HTML:
                ProcessHtmlContent(request, response);
                break;
            case TEXT_JAVASCRIPT:
            case TEXT_ECMASCRIPT:
            case APPLICATION_ECMASCRIPT:
            case APPLICATION_JAVASCRIPT:
                ProcessJavascriptContent(request, response);
                break;
            case IMAGE_JPEG:
            case IMAGE_GIF:
            case IMAGE_PNG:
                ProcessImageContent(request, response);
                break;
            case TEXT_CSS:
                ProcessCssContent(request, response);
                break;
            default:
                break;
        }
        FetcherController.Get().UrlDone(url);
    }

and then Processing HTML and fetching URLs in the given method

private void ProcessHtmlContent(HttpWebRequest request, HttpWebResponse response)
{
Logger.Log("ProcessHtmlContent: " + request.RequestUri.AbsoluteUri);
byte[] html_raw_content = ReadFullResponse(response.GetResponseStream()).ToArray();
Encoding encoding = GetResponseEncoding(response);
string html_content = encoding.GetString(html_raw_content);
string new_html_content;
current_url_ = request.RequestUri.AbsoluteUri;
Logger.Log("Fetching current url: " + current_url_);
found_urls_.Clear();
FetcherArgument[] urls = ExtractHtmlUrls(html_content, out new_html_content);
html_raw_content = encoding.GetBytes(new_html_content);
for (int i = 0; i < urls.Length; i++)
{
FetcherController.Get().FetchUrl(urls[i]);
}
WriteWebNode(request, html_raw_content);
}

and saving files using this method

private void WriteWebNode(HttpWebRequest request, byte[] contents)
{
string path = FetcherController.Get().GetPrefixDir() +
GetWebNodeFilename(request.RequestUri);
Logger.Log("Writing path:" + path + " for url:" + request.RequestUri.AbsoluteUri);
File.WriteAllBytes(path, contents);

if (index_page_)
{
File.Copy(path, FetcherController.Get().GetPrefixDir() + "index.html");
}
}

Kindly help me how can I apply progress bar ? I know there is DownloadString method of WebClient I used that one but it didnt work out.
Posted

Since HttpWebRequest doesn't supply any progress information at all, you can't. The best you're going to get is a Marquee style progress bar that just shows something is happening. You can't do the 0 to 100% bar.

The only way you can use a progress bar is if you write the request and download implementation yourself.
 
Share this answer
 
Comments
Maciej Los 5-Jan-14 10:38am    
Right, +5!
As Dave had mentioned, there is not possible to get download status from HttpWebRequest method, but WebClient enables it. Have a look here:
How to: Download a File in the Background[^]
Download Image file using WebClient with progress bar status.[^]
 
Share this answer
 
v3
Comments
Espen Harlinn 5-Jan-14 11:26am    
Nice links :-)
Maciej Los 5-Jan-14 11:33am    
Thank you, Espen ;)

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900