Click here to Skip to main content
15,917,608 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
Ok so me and my team have been working on a program for about 4 month now and everything works very fast except for the update sections.

Here is the code to what we currently have
VB
        Dim lines() = File.ReadAllLines("C:\ExcelledProducts\xAntiVirus\Database\Updates\DAILY.TXT")
        Dim DATABASE As String
        Dim readdb As New System.IO.StreamReader("C:\ExcelledProducts\xAntiVirus\Database\Updates\DAILY.TXT")
        DATABASE = readdb.ReadToEnd
        pgbAddingSignatures.Maximum = lines.Count - 1
        Dim NewSignatres As New ArrayList
        Dim tmp1 As String = Nothing

        Dim x = 0
interneal:

        Try
            If DATABASE.Contains(lines(x).ToString) Then
                pgbAddingSignatures.Value = pgbAddingSignatures.Value + 1
            Else
                pgbAddingSignatures.Value = pgbAddingSignatures.Value + 1
                DATABASE = DATABASE & vbCrLf & lines(x).ToString
            End If
            x = x + 1
            pgbAddingSignatures.Value = x
            GoTo interneal
        Catch ex As Exception

        End Try


        Dim VER = wc.DownloadString("http://excelledproducts.iwebs.ws/VER")

        My.Computer.FileSystem.DeleteFile("C:\ExcelledProducts\xAntiVirus\Database\DATABASE.EPD")

        Dim FINAL As String
        FINAL = VER & vbCrLf & DATABASE

        Dim writer As New System.IO.StreamWriter("C:\ExcelledProducts\xAntiVirus\Database\DATABASE.EPD")
        writer.Write(FINAL)
        writer.Close()

        MsgBox("Your Database Has Been Updated")


The database that we are updating is the "DATABASE.EPD", and the database where we are retrieving the new data is from the "DAILY.TXT". The "DAILY.TXT" has about 200,000 currently. Every 3 months we update the main database and reset the "DAILY.TXT". When the "DAILY.TXT" is fresh with only 20,000 lines it goes pretty fast but when we get up to about 200,000 it is very slow. Is their another way to approach how I am adding the new data to the database.
Posted

Wow, looking at your code it looks like you started vb.net with some old fashioned basic experience.
- You could use a While for looping which improves readability.
- Catch ex As Exception is hiding information

Most performance loss is in lines like these:
DATABASE = DATABASE & vbCrLf & lines(x).ToString

Each time the old string object DATABASE is copied to a new string object including the addition of a line end and a line. Very inefficient. Use a StringBuilder instead.
http://msdn.microsoft.com/en-us/library/system.text.stringbuilder.aspx#Y0[^]
For more info, check this:
Performance considerations for strings in C#[^]

Try reading the file once and in blocks to avoid overuse of memory.

Added a link, but also try google.
http://stackoverflow.com/questions/8243538/reading-and-parsing-large-delimited-text-files-in-vb-net[^]

You could also check this out:
http://www.filehelpers.com/[^]

Good luck!
 
Share this answer
 
v2
Comments
ExcelledProducts 6-Nov-12 13:22pm    
You are absolutely right. In most of my programming years I have used basic for almost 60% of my programs. We have converted the code into vb.net. Also this is just a sample code why would I release the real code to something that we are making money from. :P
ExcelledProducts 6-Nov-12 13:49pm    
Thank god for you E.F Nijboer. I looked at everything on the msdn link and now it is going very fast what used to take about 2 hours know takes less than 10 seconds. Thanks man :)
It's taken your team four months, to get to that?
Ignoring exceptions, using goto, wasting assignments... why?

Have you profiled it at all? Added any Stopwatches in order to work out what parts are slow? Tried taking out the redundant code?

To be frank, that looks like it was created by a complete beginner. Sorry if that offends, but it does.
 
Share this answer
 
Comments
ExcelledProducts 6-Nov-12 13:16pm    
This is just a sample code why would I release the real code to something that we are making money from. :P
OriginalGriff 6-Nov-12 13:59pm    
So, you don't show us the code you want us to speed up, and expect us to be able to help.
Yeah. Right.
ExcelledProducts 6-Nov-12 14:25pm    
No this code is part of the real code but the main consent is still their and I am a good enough programmer to interpret the code.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900