Click here to Skip to main content
15,920,217 members
Home / Discussions / C#
   

C#

 
AnswerRe: Obtaining a HANDLE from IO.Ports.SerialPort Pin
snorkie29-Aug-07 9:31
professionalsnorkie29-Aug-07 9:31 
QuestionHow make a dll in Visual C++ to use in C#? Pin
Rui A Sousa29-Aug-07 5:19
Rui A Sousa29-Aug-07 5:19 
AnswerRe: How make a dll in Visual C++ to use in C#? Pin
led mike29-Aug-07 6:16
led mike29-Aug-07 6:16 
GeneralRe: How make a dll in Visual C++ to use in C#? Pin
Dio2229-Aug-07 8:30
Dio2229-Aug-07 8:30 
AnswerRe: How make a dll in Visual C++ to use in C#? Pin
I Believe In GOD29-Aug-07 10:51
I Believe In GOD29-Aug-07 10:51 
QuestionHow to sort a big volume of data? Pin
michal.kreslik29-Aug-07 5:08
michal.kreslik29-Aug-07 5:08 
AnswerRe: How to sort a big volume of data? Pin
snorkie29-Aug-07 8:38
professionalsnorkie29-Aug-07 8:38 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik29-Aug-07 8:53
michal.kreslik29-Aug-07 8:53 
Sure, I'll be thankful for any help.

Basically, I've got txt files that contain information about Forex price ticks. Each row contains this information that I'm interested in:

- date/time
- price

Now the txt files overlap themselves in such a way that the ending part of one file contains exactly the same ticks as the beginning part of another file (this was to ensure that during the preceding export, no price ticks were omitted). Obviously, this is not true neither for the very first nor for the very last file.

The price ticks within a single file are guaranteed to not be duplicate.

If we want to get a continuous data stream that would only contain unique ticks (rows), the simple solution would be to load all rows from all files and sort the rows on: DateTime ASC, FileID ASC, RowNumber ASC.

Then I could go through all sorted rows and remove those that contain the same DateTime as the last valid DateTime, but with a new FileID. In other words, only ticks with the same FileID can have the same DateTime to ensure there are no duplicates.

Unfortunately, the DataTable object is throwing an OutOfMemoryException on me if I attempt to DataTable.Select() such a big chunk of data (about 15 million rows). The same happens with DataTable.DefaultView.Sort.

It works on smaller data sample with no memory exceptions being thrown, though.

Any ideas?

Thanks much for any input!
Michal
GeneralRe: How to sort a big volume of data? Pin
snorkie29-Aug-07 9:06
professionalsnorkie29-Aug-07 9:06 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik29-Aug-07 9:36
michal.kreslik29-Aug-07 9:36 
GeneralRe: How to sort a big volume of data? Pin
Spacix One29-Aug-07 10:30
Spacix One29-Aug-07 10:30 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik29-Aug-07 19:32
michal.kreslik29-Aug-07 19:32 
GeneralRe: How to sort a big volume of data? Pin
Spacix One30-Aug-07 4:28
Spacix One30-Aug-07 4:28 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik30-Aug-07 4:43
michal.kreslik30-Aug-07 4:43 
GeneralRe: How to sort a big volume of data? Pin
Thomas Krojer2-Sep-07 22:46
Thomas Krojer2-Sep-07 22:46 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik3-Sep-07 6:06
michal.kreslik3-Sep-07 6:06 
GeneralRe: How to sort a big volume of data? Pin
Thomas Krojer3-Sep-07 23:49
Thomas Krojer3-Sep-07 23:49 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik4-Sep-07 0:10
michal.kreslik4-Sep-07 0:10 
GeneralRe: How to sort a big volume of data? Pin
Thomas Krojer18-Sep-07 23:00
Thomas Krojer18-Sep-07 23:00 
GeneralRe: How to sort a big volume of data? Pin
michal.kreslik18-Sep-07 23:17
michal.kreslik18-Sep-07 23:17 
GeneralRe: How to sort a big volume of data? Pin
Thomas Krojer19-Sep-07 0:31
Thomas Krojer19-Sep-07 0:31 
Questiondynamically creating forms from db Pin
algoaddict29-Aug-07 4:53
algoaddict29-Aug-07 4:53 
Questionword document to xml document Pin
sk840529-Aug-07 4:43
sk840529-Aug-07 4:43 
AnswerRe: word document to xml document Pin
led mike29-Aug-07 5:10
led mike29-Aug-07 5:10 
JokeRe: word document to xml document Pin
Spacix One29-Aug-07 8:28
Spacix One29-Aug-07 8:28 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.