Very faster way to filter all emails end with a special caractere in a big text file using C#

Question

1.00/5 (2 votes)

See more:

Hello ,

i want a very faster way to filter all emails that end with a special caracter in a big text file (more than 200MB) using c#

What I have tried:

var lines = File.ReadAllLines(file path);

foreach(var line in lines)
{
if (line.EndsWith(myWord))
{
outputEmails.Text += line + Environment.NewLine;
}
}

Posted 25-Apr-17 15:33pm

dravoss

Updated 25-Apr-17 16:10pm

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Dave Kreskowiak · Answer 1 · 2017-04-25T16:10:00

Well, you can make it a tiny bit faster by replacing that string concatenation with a StringBuilder. Why? Because strings in .NET are immutable. Once created you cannot change them. What you're doing with that concatenation is creating a new string object every time you append something to it.

You could also probably pickup a very minor speed improvement by replacing the .ReadAllLines with a StreamReader and processing the file line by line one at a time. You'll also save yourself 200MB of memory and possibly some page swapping.

If you're looking for a single character at the end of each line, you might be able to eek out another boost by treating each line, which will be a string, as an array of char and specifically look at the last index in the array for that character. This will avoid some of the overhead of the .EndsWith method. You're writing something that specializes in looking at the last character instead of using the more general use case that looks for an entire string of characters.

Other methods to make it faster involve taking the time to pre-process the text file and create an index. This will take the amount of time your taking now to build the index. Then it's easy enough and much faster to get the lines that you need by using the index. But, I don't think that's what you're talking about.