GetFiles Limited number of files

Question

3.00/5 (2 votes)

See more:

I have a web service which reads the content of a directory which contains 40 000 files so each time the service calls GetFiles it takes ages.

My question is it possible to request a number of files, for something like GetFiles(Path, StartIndex, Count)

Posted 11-Jan-11 6:21am

arrie1610

Add a Solution

4 solutions

Solution 3

why not use DirectoryInfo.EnumerateFiles ?

Posted 11-Jan-11 6:46am

cechode

Comments

arrie1610 11-Jan-11 13:21pm

Would this not fecth all the files anyway if I am looking for the last 10 files in the directory?

Solution 4

Could you have a utility map each file name/path to a database table, then load the names of the first / last / middle x number of files, depending on your criteria?

[Update]
Is file indexing turned on for the drive?

Posted 11-Jan-11 8:04am

wizardzz

Updated 11-Jan-11 10:38am

v2

Comments

wizardzz 11-Jan-11 15:12pm

Hmmm, I'm curious as to the 1 vote? In my experience, querying a 40k row db for the last 10 entries is not slow.

wizardzz 11-Jan-11 16:36pm

Is there any explanation for the down vote? Nope? Interesting... And this down vote must have come from someone with quite a bit of authority as I've never had a down vote have this much effect on points.

Solution 5

Good question: I'll vote 5.

You may need P/Invoke to use WIN32.

MIDL

// The CharSet must match the CharSet of the corresponding PInvoke signature
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)]
struct WIN32_FIND_DATA
{
    public uint dwFileAttributes;
    public System.Runtime.InteropServices.ComTypes.FILETIME
        ftCreationTime;
    public System.Runtime.InteropServices.ComTypes.FILETIME  
        ftLastAccessTime;
    public System.Runtime.InteropServices.ComTypes.FILETIME 
        ftLastWriteTime;
    public uint nFileSizeHigh;
    public uint nFileSizeLow;
    public uint dwReserved0;
    public uint dwReserved1;
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst=260)]
    public string cFileName;
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst=14)]
    public string cAlternateFileName;
}

[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
static extern IntPtr FindFirstFile(string lpFileName,
    out WIN32_FIND_DATA lpFindFileData);

[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
static extern bool FindNextFile(IntPtr hFindFile,
       out WIN32_FIND_DATA lpFindFileData);
}

You see? Using FindFirstFile you find only one file, then you work with FindNextFile using a pointer returned by FindFirstFile.
Unfortunately, I don't see the how to implement StartIndex. However, you can think of some combination of this method with some renaming schema.

Posted 11-Jan-11 11:46am

Sergey Alexandrovich Kryukov

Updated 11-Jan-11 11:52am

v2

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

#realJSOP · Accepted Answer · 2011-01-11T06:26:00

Solution 1

Not unless you write the code yourself. Of course, you could create a thread that makes the call, and let the app go on about its business.

EDIT for your response ==============

I kinda figured that the user is waiting for the results, but by using a thread, at least the user can cancel the operation if he wants to...

I found this Codeproject article with google.

A Faster Directory Enumerator[^]

Posted 11-Jan-11 6:26am

#realJSOP

Updated 11-Jan-11 7:41am

v4

Comments

Sergey Alexandrovich Kryukov 11-Jan-11 12:44pm

Great point! - a 5.

#realJSOP 11-Jan-11 13:39pm

FROM OP: Unfortunately the user is waiting for the response, thanks anyway.

I am thinking one posibility would be rename all the files with a suffix to the filenames to group them into a smaller number of groups then to call GetFiles(Filename[Suffix].*)

Will this work or would the processing be the same?