How to asyn more than one httprequest at same time

Question

0.00/5 (No votes)

See more:

hello and good day programmers,
i have a problem that is kind of difficult to solve on my own. what i want to achieve is getting news from different source and then updating the observableCollection with the returned json. the probem now is that i am getting the request one by one and then updating the observableCollection which i used as binding to my listview but it makes the program do heavy works and i dont like the way it updates in the listview.
so please what is the best way to get the news from different sources and added to the listview at same time or what is the best way to achieve this..
i am making more than one httprequest which i guess is not the right step to take. please help

What I have tried:

using (var httpClient = new HttpClient())
                    {
                        httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "application/json");
                        httpClient.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; 
                        rv:19.0) Gecko/20100101 Firefox/19.0");
                        response2 = await httpClient.GetAsync("https://website.com/feed/");
                    }
                        string stud6 = await response2.Content.ReadAsStringAsync().ConfigureAwait(false);
                        Article obj6 = JsonConvert.DeserializeObject<Article>(stud6);
                    // _Feeds.Clear();
                     foreach (var kvp in obj6.items)
                    {
                       
                        var feed = new ArticleModel()
                        {
                            Title = kvp.title,
                            pubDate = kvp.pubDate.Replace("+0000", ""),
                            Link = kvp.link,
                            Description = kvp.description,
                            Content = kvp.content,
                            Author = kvp.author,
                            Thumbnail = kvp.thumbnail,
                        };
                        _Feeds.Add(feed);
                        //await RefreshDB();
                    }

<pre>using (var httpClient = new HttpClient())
                    {
                        httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "application/json");
                        httpClient.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; 
                        rv:19.0) Gecko/20100101 Firefox/19.0");
                        response2 = await httpClient.GetAsync("https://website.com/feed/");
                    }
                        string stud6 = await response2.Content.ReadAsStringAsync().ConfigureAwait(false);
                        Article obj6 = JsonConvert.DeserializeObject<Article>(stud6);
                    // _Feeds.Clear();
                     foreach (var kvp in obj6.items)
                    {
                       
                        var feed = new ArticleModel()
                        {
                            Title = kvp.title,
                            pubDate = kvp.pubDate.Replace("+0000", ""),
                            Link = kvp.link,
                            Description = kvp.description,
                            Content = kvp.content,
                            Author = kvp.author,
                            Thumbnail = kvp.thumbnail,
                        };
                        _Feeds.Add(feed);
                        //await RefreshDB();
                    }

<pre>using (var httpClient = new HttpClient())
                    {
                        httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "application/json");
                        httpClient.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; 
                        rv:19.0) Gecko/20100101 Firefox/19.0");
                        response2 = await httpClient.GetAsync("https://website.com/feed/");
                    }
                        string stud6 = await response2.Content.ReadAsStringAsync().ConfigureAwait(false);
                        Article obj6 = JsonConvert.DeserializeObject<Article>(stud6);
                    // _Feeds.Clear();
                     foreach (var kvp in obj6.items)
                    {
                       
                        var feed = new ArticleModel()
                        {
                            Title = kvp.title,
                            pubDate = kvp.pubDate.Replace("+0000", ""),
                            Link = kvp.link,
                            Description = kvp.description,
                            Content = kvp.content,
                            Author = kvp.author,
                            Thumbnail = kvp.thumbnail,
                        };
                        _Feeds.Add(feed);
                        //await RefreshDB();
                    }

Posted 7-Nov-22 16:32pm

sergeibystrov

Updated 7-Nov-22 17:52pm

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Graeme_Grant · Answer 1 · 2022-11-07T17:52:00

Here is a DownloadService with a batching mode (10 at a time) for grabbing files, like images.

C#

public class DownloadService
{
    public string SavePath { get; set; }
    private IHttpClientFactory _httpFactory { get; }

    public DownloadService(IHttpClientFactory httpFactory)
        => _httpFactory = httpFactory;

    public async Task<bool> DownloadToFileAsync(string url, string filePath)
    {
        Console.WriteLine($"Downloading: {url} as {filePath}");
        filePath = Path.Combine(SavePath, filePath);

        var request = new HttpRequestMessage(HttpMethod.Get, url);
 
        var client = _httpFactory.CreateClient();
        var response = await client.SendAsync(request);

        if (response.IsSuccessStatusCode)
        {
            try
            {
                await using Stream webStream = await response.Content.ReadAsStreamAsync();
                await using FileStream fileStream = new FileStream(filePath, FileMode.Create, FileAccess.Write,
                    FileShare.None,
                    bufferSize: 64 * 1024, useAsync: true);

                await webStream.CopyToAsync(fileStream).ConfigureAwait(false);

            }
            catch (Exception ex)
            {

                throw;
            }
            
            return true;
        }

        Console.WriteLine($"StatusCode: {response.StatusCode} for {url}");
        return false;
    }

    public async Task<bool> BulkDownloadToFileAsync(Dictionary<string, string> jobs)
    {
        // note: this is considered spamming requests, not a good idea if a single website/server...
        //    return (await Task.WhenAll(jobs.Select(job
        //                => DownloadToFileAsync(job.Value, job.Key)))
        //            .ConfigureAwait(false))
        //        .All(x => x);

        // note: a better option is to batch requests
        var batches = jobs.Chunk(10);

        foreach (var batch in batches)
        {
            bool success = (await Task.WhenAll(batch.Select(job => DownloadToFileAsync(job.Value, job.Key)))
                    .ConfigureAwait(false))
                .All(x => x);

            if (!success)
                Debugger.Break();

            // pause before continuing
            await Task.Delay(200).ConfigureAwait(false);
        }

        return true;
    }
}

To use:

C#

var builder = new HostBuilder()
    .ConfigureServices((hostContext, services) =>
    {
        services.AddHttpClient();
        services.AddTransient<DownloadService>();
    }).UseConsoleLifetime();

var host = builder.Build();

using var serviceScope = host.Services.CreateScope();

var services = serviceScope.ServiceProvider;

var jobs = new Dictionary<string, string>
{
    ["local_save_filename"] = "remote_url_to_file",
    ["local_save_filename"] = "remote_url_to_file",
    ["local_save_filename"] = "remote_url_to_file",
    // etc...
};

var downloadService = services.GetRequiredService<DownloadService>();
downloadService.SavePath = @"path_to_save_downloads";

var success = await downloadService
    .BulkDownloadToFileAsync(jobs)
    .ConfigureAwait(false);

Notes:
* Best practice is to use the HttpClientFactory when working with the HttpClient - ref: HttpClient guidelines for .NET - .NET | Microsoft Learn[^]
* The above code is for demonstration purposes only and is not production ready.

I am sure that you get the idea and can adapt to your needs.

Hope this helps!