Click here to Skip to main content
15,897,891 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
my app uses a list to store some urls and multi threading to download each url. The problem is that in current method all threads are created at once. I want to be able to use 25 threads, then wait for them to finish and then use another 25 threads. Now the question is how do i check whether the first 25 threads are completed ?

VB
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        ProgressBar1.Maximum = ListBox1.Items.Count
        For i As Integer = 0 To ListBox1.Items.Count - 1
            Dim t As New Threading.Thread(AddressOf dwnld)

            t.Start(ListBox1.Items(i).ToString)
        Next

    End Sub


VB
Public Sub dwnld(ByVal url As String)
        Try
            Dim result As String = String.Empty

            Dim txt As String = ""

            Dim request As HttpWebRequest = DirectCast(WebRequest.Create(url), HttpWebRequest)

            request.Method = "GET"

            Using stream = request.GetResponse().GetResponseStream()
                Using reader = New StreamReader(stream, Encoding.UTF8)
                    result = reader.ReadToEnd()
                End Using
            End Using

            Dim doc As New HtmlAgilityPack.HtmlDocument()

            doc.Load(New StringReader(result))

            Dim root As HtmlNode = doc.DocumentNode

            For Each link As HtmlNode In root.SelectNodes("//p")
                Dim att As String = link.InnerText.Trim
                txt = txt & att & vbCrLf
            Next

            Me.settext(txt)
        Catch ex As Exception

        End Try
    End Sub


i want to do a batch of 25 urls, wait for them to finish and then pass next 25 urls. The current method uses all 100 at once. Any help please?
Posted
Updated 27-Nov-11 21:30pm
v2
Comments
DaveAuld 28-Nov-11 4:21am    
Why do it in batches, why not just set up a number of threads that keep pulling a URL from the list until the list is complete? That way your threads are always kept busy and your app will be more efficient.
amit_upadhyay 28-Nov-11 5:14am    
yeah even that would work. How do i do that? Could you point me to some link or code plz? I mean after a thread has completed its work do i just take another url from list?
Albert Holguin 8-Jan-12 1:07am    
You just need to make your threads access the items from an array (or list, or vector, or whatever) and control access to it using a mutex.

1 solution

you can control the threads like this:

set a timer tick with 1 second update.

start every thread with a PID defined (you can do that correctly if you get the current PID of windows and select max, with this, you can set other differents at this moments.
get a list of this PID for use it after (only the PID you start in threads).
when you start all the 25 threads you can start the timer tick.

with this list of PID, control it with the timer tick, if the thread with the PID xxx exists or not, count it until the conter equals zero.

and if the count equals zero, you can start another threads as you want.

i hope this is useful to you.

regards.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900