|
Hi David,
Thanks for replying. I'm trying to shy away from vector usage because, due to the large amount of data, vector storage and usage (particulary push_back) are killing the efficiency. So I'm trying to experiment with other methods such as map of maps.
|
|
|
|
|
Hi,
std::vector::push_back is only a problem if the internal storage for the vector has to change. If you do a std::vector::resrve to an estimate of the the maximum size of the vector before doing anything else it'll speed up push_back no end.
On a related point, lookup in a map of maps with loads of entries is going to be pretty slow...
Cheers,
Ash
|
|
|
|
|
b-rad311 wrote:
Thanks for replying. I'm trying to shy away from vector usage because, due to the large amount of data, vector storage and usage (particulary push_back) are killing the efficiency. So I'm trying to experiment with other methods such as map of maps.
maps actually begin to be inefficient at large numbers too. Remember that a map is internally stored as a binary tree in most implementations. This is highly efficient method of searching since it always becomes the equivalent of a quick search through your data to find any one node. However, the depth of the tree begins to slow down all operations as well. As mentioned, if you reserve space on a vector you can achieve high speed. Vector reallocates at 50% above your last block size when it runs out of memory, then shifts the contents over to a new array that is larger. It is the reallocation and move that occupies most of the time of the push_back.
There are also alternatives, depending on what you want to achieve. I use pre-allocated vectors as large hash-tables, with the appropriately speedy hash algorithm you can beat a map on very large data sets. It all comes down to need and desired performance.
so you spoke of a time key, but what are you storing and how do you need to access it?
_________________________
John Andrew Holmes "It is well to remember that the entire universe, with one trifling exception, is composed of others."
Shhhhh.... I am not really here. I am a figment of your imagination.... I am still in my cave so this must be an illusion....
|
|
|
|
|
Hello,
Thanks for replying. I am storing forces and moments (so double or long doubles) for an engineering application. At times, there could be as many as 1 million entries that need to be stored. I need to be able to quickly locate zeroes and eliminate them along with all other values in the corresponding "rows".
modified on Thursday, May 27, 2010 9:56 AM
|
|
|
|
|
b-rad311 wrote: Thanks for replying. I am storing forces and moments (so double or long doubles) for an engineering application.
You definitely don't want a map of maps because it doesn't sound like you have two indexes. You could use a map of a structure, or pair, and use the one index of time. However... at one million entries you are going to start noticing the significant hit of the search through the map.
start with a quick review of performance considerations on STL:
GDC 2001 Round Table Report[^]
Custom STL Allocators (Power Point Presentation)[^]
Common Performance Mistakes in Games (Power Point Presentation)[^] (this is my favorite)
Ignore that the latter is for the Xbox, the issues are the same. Games require performance, performance on STL containers are a common problem, but usually because of a misunderstanding of how they work. When do you make your own? why? how? do you leverage the existing STL work when you do? (yes!!)
If your time is sampled at a constant rate you directly turn time into an index, so I would use a vector, but I would pre-allocate memory with reserve such that the allocation occurs only once at the beginning and there after, random access direct memory speed will beat anything hands down.
If you sample rate is not constant, and you need random access over linear reading, you can consider an STL form of a Hash when you get into a large data sets. There are still two possibilities: one-time hash, or bucket hash. One-time hashes are hard to set up, usually meaning you allocate more memory than you need by 50% in a vector, then generate an index from your time using a mathematical hash function which converts your key directly to an index value. If you choose your hash function well for your key-type, this works well, but requires more memory. If you choose your hash function poorly, your tend to reallocate when you hit duplicate keys to the a given index and there goes your performance down the tubes again.... A bucket hash is kind of compromise. You use a smaller vector as the hash, but duplicates are stored in individual lists or maps (the buckets).
The bad news is that hashes do not have a really good way of accessing the values in order, it is strictly a random access method. If you need linear reads, I would go with a reserve value on a vector to improve vector performance. You can also do the same with a map, but maps allocate blocks as you go, without recycling old allocations, so it usually isn't necessary.
I used to have a table of STL performance values for each key function (push back, push front, insert, find, iterators, etc), but I can't seem to find it with google at the moment and I have to jump back to my work for a while. Think over what you have, how you intend to read it, how you can read it too.
Good luck!
_________________________
John Andrew Holmes "It is well to remember that the entire universe, with one trifling exception, is composed of others."
Shhhhh.... I am not really here. I am a figment of your imagination.... I am still in my cave so this must be an illusion....
|
|
|
|
|
El Corazon,
Thanks for all the info.
|
|
|
|
|
b-rad311 wrote: I need to be able to quickly locate zeroes and eliminate them along with all other values in the corresponding "rows".
okay, I am back... sorry, but work comes first. Anyhow, tell me about this bit. So you are looking for any time that has zero in one of the other data items? so you are trying to find incomplete data sets and remove that time-period such that you have only complete data sets in the final composition of data stored? When you finally remove all the incomplete pieces are you then going to process the data in time series? will you be doing any random access by time?
if you have no requirement for random access I would do a cull cycle on a list. A list is good for sequential access and horrible for any find() or random access method, but good on add/delete/insert.
iteration one add all data to the list using time as your key....
iteration two using iterators, sequentially check data sets for zero values, remove from list.
iteration three using iterators, sequentially do your processing of final dataset (this is an assumption on my part and may be wrong)
if you need random access, then yes, we are probably back to maps, vectors handle add/deletes poorly, though if you are smart in your methods, and you have a way of turning time into an index you can do it. I'll go over both ways using methods I have done.
map/multimap: (assumption: need random access, and sequential, and add/delete) (maps assume unique keys, multimaps assume duplicate keys exist, multimaps perform even worse)
iteration one add all data to the map using time as a key
iteration two using iterators, sequentially process data sets for zero values, remove from map/multimap
iteration three using find() and iterators, do what ever processing is necessary
vector: (assumption: key can be turned into linear index, needs random access, no insert or push_front, only push_back)
(key assumption is index, recording data at 60 times a second means that index = floattime*60)
setup reserve() necessary space to prevent allocation time loss
iteration one add all data to vector with push_back only
iteration two flag all structures as discarded that contain zeros
iteration three process using random/sequential access combination (ignoring marked items)
if all you ever do is sequential access you need only a list which has near the power of vector in sequential access, but finds are horrible time-wasters in lists....
_________________________
John Andrew Holmes "It is well to remember that the entire universe, with one trifling exception, is composed of others."
Shhhhh.... I am not really here. I am a figment of your imagination.... I am still in my cave so this must be an illusion....
|
|
|
|
|
Something I thought of when answering someone else's question, but it might be of use to you too. Have you considered a skip-list instead of a map? maps are binary trees, and skip lists occasionally offer speed improvements. It all comes down to need and use, but it is worth thinking about: http://www.codersource.net/microsoft-net/c-advanced/skip-list-a-simpler-alternative-to-binary-trees.aspx[^]
_________________________
John Andrew Holmes "It is well to remember that the entire universe, with one trifling exception, is composed of others."
Shhhhh.... I am not really here. I am a figment of your imagination.... I am still in my cave so this must be an illusion....
|
|
|
|
|
The brute force way is:
a[ 0.0 ][ 1.57 ] = 2.65;
Ad nauseum...
There might be a quicker way using the std::pair constructor in map, but I can't quickly see a clean way of avoiding trashing the stored maps if more than one entry has the same primary key.
Cheers,
Ash
|
|
|
|
|
Thanks Ash.
Would this method of loading hold true even if there were 50 columns? Also, there shouldn't be an opportunity for duplicate keys.
|
|
|
|
|
Hi,
How to check system windows SDK version?
|
|
|
|
|
Check which one you installed! SDK is Software Development Kit and is released independently of Windows. It is usually accompanied by a new compiler, and will often include functionality that is not supported on earlier versions of Windows. However you should check the MSDN pages for compatibility issues on particular functions.
It's time for a new signature.
|
|
|
|
|
I have showing a CListCtrl inside a CView.
I am adding the items using DrawItem method of CListCtrl.
When I open the view, Only one item gets added inside the List control. When I minimize or maximize the view then one by one the items are getting added.
How to correct the problem.
Please note that the items that are getting added in the list control are dynamic values which will get changed from time to time. So the changed values are to be reflected in the list control
Please help me
|
|
|
|
|
Not sure what you mean but my guess is that you don't know how to make your list control update when your values change, right? I'd say, when a value changes, call RedrawItems[^] (if you know which item belongs to which value), or simply do a RedrawWindow[^] on your list view. If your values change a lot then redrawing the list every time becomes very unefficient so you could add some delay to the updates (like, update your list only if 5 seconds has passed since the last time you redrawn it and a value has changed or somesuch).
If you do not know when exactly your values change then i guess you have to rely on polling, meaning, check the values frequently from a timer or somesuch and if you detect a change, do the drawing.
> The problem with computers is that they do what you tell them to do and not what you want them to do. <
> Computers are evil, EVIL i tell you!! <
|
|
|
|
|
Thanks for your support.
I am Showing the List Control inside my view class using the below code.
pAlrmListCtrl is the list control object.
In the DrawItem() function of the listcontrol I am adding the items (approx 10 items) to the list control. When I run the application only one item shows up in the list control. When I maximize and minimize the view (CMYView), the items are getting added one by one for each maximize or minimize operation.
Also when i Set break point in the DrawItem() function, the list control shows up all the items.
It seems like a refresh problem
CMYView::OnDraw()
{
// TODO: add draw code here
CRect rect;
CExperionDoc* pDoc =(CExperionDoc*)GetDocument();
GetClientRect( &rClientArea );
pAlrmListCtrl->rAlrmScreen = rClientArea;
sWindowTitle = "Process Alarm";
pDoc->SetTitle(sWindowTitle);
pAlrmListCtrl->UpDateList();
Invalidate(TRUE);
}
Kindly help me to get over this problem.
|
|
|
|
|
Why do you add items to your list where you should draw the view??
> The problem with computers is that they do what you tell them to do and not what you want them to do. <
> Computers are evil, EVIL i tell you!! <
|
|
|
|
|
I am showing the Clistctrl inside my CMYview
|
|
|
|
|
Ok, but as the documentation[^] says, OnDraw is getting called whenever your view needs to be drawn (or printed). Why do you need to fill your list every time your view is drawn? Sounds like bad design to me unless there is something obvious i am missing...
> The problem with computers is that they do what you tell them to do and not what you want them to do. <
> Computers are evil, EVIL i tell you!! <
|
|
|
|
|
manoharbalu wrote: I have showing a CListCtrl inside a CView.
So are you using CListView?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
|
|
|
|
|
Hi,
Im my dialog i have one CStatic control.Im using that to display one float varialbe and also string.I assigned float varialbe(m_fPV) and control varaible(m_cPV).
When i give float value it displays correct value.But when i set text,it doesnot display.it shows 0 only.
My coding:
DDX_Text(pDX, IDC_PVVAL, m_fPV);
DDX_Control(pDX, IDC_PVVAL, m_cPV);
if(oTagBase->GetTagType()==CONT)
m_fPV = oTagBase->GetPV();
if(oTagBase->GetTagType()==SWITCH)
m_cPV.SetWindowText("OPEN");
Anu
|
|
|
|
|
Probably there's a UpdateData somewhere that overrides your SetWindowText.
IMO skip the DDX_Text thingy.
Just use a CString, Format the text and SetWindowText it to the control.
Watched code never compiles.
|
|
|
|
|
The first assumption I'm going to make is that that block of code is in your DoDataExchange implementation. Would something a bit simpler work?
if( oTagBase->GetTagType()==CONT )
{
DDX_Text( pDX, IDC_PVVAL, m_fPV );
}
else if( oTagBase->GetTagType() == SWITCH )
{
CString text( "OPEN" );
DDX_Text( pDX, IDC_PVAL, text );
}
The advantage of this is it keeps everything consistent - you're only using one mechanism to exchange data between the edit control and the members of the dialogue box. And it removes the need to subclass anything.
[Another thing you could do is to move the comparisons into whatever class oTagBase belongs to, then you could write something like:
if( oTagBase->finite_resistance() )
{
DDX_Text( pDX, IDC_PVVAL, m_fPV );
}
else if( oTagBase->infinite_resistance() )
{
CString text( "OPEN" );
DDX_Text( pDX, IDC_PVAL, text );
}
]
Cheers,
Ash
|
|
|
|
|
Hi,
How to crop an image and save that part only.
I think this might be possible using gdi++ but I could not find.
Help needed...
|
|
|
|
|
john5632 wrote: I think this might be possible using gdi++ but I could not find.
It's all here[^] in the Bitmap class.
It's time for a new signature.
|
|
|
|
|
i am trying to load a texture file in open gl in visual c++ but i donot know that where should i keep the texture file so that i can load it in my program.can any one tell me???
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
