That is always a PITA - string sort order is different from numeric sort order, so you can't directly sort by extension the way you want, even if it contains only numeric digits.
Have a look at this:
Numeric String Sort in C#[
^] - it sorts the "wrong" part of the file name for you, but it should be pretty easy for you to change!