DataTable performance issue. - C# Discussion Boards

Your previous posters point is here!

foreach (DataRow myDataRow1 in myDataSet.Tables["points"].Rows)
{
myDataSet.Tables["points"].Rows[0]["track_easting"] = "665544";
}

You are iterating through all rows, but only updating the first row(ie Rows[0])

try

foreach (DataRow myDataRow1 in myDataSet.Tables["points"].Rows)
{
myDataRow1["track_easting"] = "665544";
}

Sweet .. that is something that I am going to try. Luc ( in the previous reply ) pointed me in exactly the same direction, So after a little bit of tinkering, I came up with this :

for (int x = 0; x <  myDataRow.Table.Rows.Count; x++)<br />
                {<br />
                    myDataSet.Tables["points"].Rows[x]["track_easting"] = "665544";<br />
                }

Is there a significant difference between the two ?

Kind regards,

Sure, readability is much better with foreach, as pprice has shown;
and readability or clarity leads to better maintainability.
Performance wise I dont expect a significant difference.

Smile | :)

Luc Pattyn

[My Articles] [Forum Guidelines]

The main differenc eappears to be your understanding of foreach. In your new version you are using a for each loop which is fine, but for simplicity you could revert to the for each loop

here is my original example

foreach (DataRow myDataRow1 in myDataSet.Tables["points"].Rows)
{
myDataRow1["track_easting"] = "665544";
}

Here is your original example

foreach (DataRow myDataRow1 in myDataSet.Tables["points"].Rows)
{
myDataSet.Tables["points"].Rows[0]["track_easting"] = "665544";
}

The difference between the two is that I am using the row populated by the foreach statememnt, you are using foreach to iterate through the list, but then calling the update directly on the table.

The foreach command in this case will step through every row in the table returning a reference to the row as myDataRow1, which can be directly altered. The benefit is simply not having to work out how many objects are there etc as per a for loop.

look foreach up, it is useful and as per another previous post, easy to read. whats more it is type safe.

Hope it helps

How to Make rdlc report to Rum Time
I have some problems to make a rdlc report
Simply
i Want make a rdlc report thorw Coding;

Thanks

Could someone point me at the risks of embedding forms from one development invironment into other development environments? For example writing an application in python and embedding it into a windows application or vice versa and the two apps have to communicate with each other.

I have been tasked with writing a windows app with a composite Gui which will have a COM facade and then have it embedded into an Omnis Studio form. For those of you unfamiliar with Omnis Studio think of Visual Studio on acid.

What are the dangers of writing an application in C# then embedding it onto another development platforms form?

daviiie wrote:
Could someone point me at the risks of embedding forms from one development invironment into other development environments?

I don't know. But the Eiffel guys have done this. You might like to consult them. See:

Full Eiffel on the .NET Framework[^]

About 2/3 of the way down in the section on Eiffel libraries there is a screenshot of a Win Forms datagrid running inside a native EiffelVision window.
It says:

"Particularly interesting is the ability to combine Eiffel mechanisms, such as EiffelVision, with .NET Framework mechanisms, such as Windows Forms. For example, you can embed, in a possibly complex EiffelVision application, an advanced Windows Form control such as a Datagrid providing direct display of a database through ADO.NET. The figure below shows such a Datagrid displayed as part of an EiffelVision window."

Also, can't you do the same with MFC - embed Windows Form in an MFC window? Still, the Eiffel example is interesting in that it illustartes doing this with a non-MS host.

Kevin

Hi,

I needed to add 80,000 records. I do following, see following pseudo:

1. While(Readline != EOF)
2. Create a row
3. Add the Row to the DataTable (this starts slowing down after 30,000 records, and goes completely blank around 80th,000 record)
3. End While
4. Dump the DataTable in to MS Access database.

Although I am using a background worker thread, but all is in vain.

public void RunThd(object theFile)<br />
{<br />
<br />
Thread thd = new Thread(Go);<br />
<br />
thd.IsBackground = true;<br />
<br />
thd.Start(theFile);//Starting the thread, passing the StreamReader as an argument.<br />
<br />
}

Following code is working OK. Reads the StreamReader, prepares Rows, and dumps the DataTable into the MYTABLE1 table.

void Go(object theFile)<br />
<br />
{<br />
<br />
StreamReader Reader = (StreamReader)theFile;<br />
<br />
string sqlSelect = "SELECT * FROM MYTABLE1";<br />
<br />
string sqlConnectionString = System.Configuration.ConfigurationManager.ConnectionStrings[@"conString"].ConnectionString;<br />
<br />
try<br />
<br />
{<br />
<br />
DataTable dTable = new DataTable();<br />
<br />
OleDbConnection oleConnection = new OleDbConnection(sqlConnectionString);<br />
<br />
OleDbDataAdapter oleAdapter = new OleDbDataAdapter(sqlSelect, oleConnection);<br />
<br />
OleDbCommandBuilder cmdBuilder = new OleDbCommandBuilder(oleAdapter); //Command Builder<br />
<br />
oleAdapter.Fill(dTable);<br />
<br />
string strLine = stReader.ReadLine();<br />
<br />
//Prepare all of the Rows - around 80,000 records.<br />
<br />
DataRow thisRow = null;<br />
<br />
while (!Reader.EndOfStream)<br />
<br />
{<br />
<br />
strLine = stReader.ReadLine();<br />
<br />
//Add rows<br />
<br />
thisRow = dTable.NewRow();<br />
<br />
 <br />
<br />
thisRow["Col1"] = strLine.Substring(0, 1);<br />
<br />
thisRow["Col2"] = strLine.Substring(1, 11);<br />
<br />
thisRow["Col3"] = strLine.Substring(11, 1);<br />
<br />
dTable.Rows.Add(thisRow);//after around 30000 records, app seems slower.<br />
}<br />
<br />
//Update database<br />
oleAdapter.AcceptChangesDuringFill = true;<br />
oleAdapter.Fill(dTable);<br />
<br />
oleAdapter.Update(dTable);//updates the table in ms access database. Is it possible to refresh DataTable after 10,000 records? Just to mitigate the performance issues? Add 10,000 records, then refresh DataTable, then Add another 10000 and refresh, and so on for the rest of the records.<br />
<br />
}<br />
<br />
catch (Exception e)<br />
{<br />
MessageBox.Show(e.Message);<br />
}<br />
<br />
}<br />
<br />
}

Problem with the code above:

Performance Issues. The app goes down. System starts getting slow, etc. Although, I am using a Background thread, and not using ExecuteNonQuery() that will make app more slower. All that is going good, is really not so well.

Resolution:

Things that I was thinking are:

1. Dump every 10,000 records, and then refresh the DataTable, then fill next 10,000 and dump into the database, and so on so forth. But how to achieve this, needs a Flood Light to be thrown upon (0:

2. Cannot think of any other option?

Thanks for reading all of it. I'd appreciate ideas, helps, comments, suggestions, codes, pointers, etc. Bottomline, need help resolving the performance issue.

How much memory do you have in your PC that you are running this on? If things are slowing down it seems to me that you are running out of memory. It doesn't really make sense you are having problems since you are only taking 13 bytes of memory for each row. So even at 80,000 records that doesn't take up that much memory.

Here is my idea. First you can try adding some code like dTable.AcceptChanges
after the Add rows line.

If that doesn't help you could try a totally different approach. The basic xml of a dataset is:
<newdataset>
<dTable>
<column1>1</column1>
<column2>1234567890</column2>
<column3>1</column3>
</dTable>
<dTable>
<column1>2</column1>
<column2>2345678901</column2>
<column3>2</column3>
</dTable>
<dTable>
<column1>3</column1>
<column2>3456789012</column2>
<column3>3</column3>
</dTable>
</newdataset>

If you wrote out a file and saved it, you could then load a dataset with that data. Of course if you have a memory issue, that is not a long of memory this is going to take more memory then your other solution.
If you saved this file you can load it into a dataset, so the code would be something like:
DataSet ds = new DataSet();
ds.Load(@"file.xml");

Assuming you saved the xml above to file.xml

Anyway, just another idea.

Ben

Thanks for the idea Ben. I appreciate. Isint there anythign like Bulk Insert of flat files?

Not that I know of. I think when you are using flat files you are limited with what you can do.

Ben

I appreciate your replies Ben.

Thanks.

DO you have indexes on the table in the database? Drop the indexes, do the import, recreate the indexes. Fixed a similar problem I had some time ago.

Regards

Angel
*********************************************

Thanks for your reply Angel.

No, there are no indexes. This is just a plain flat file. I have no issues with the process that I am following, except for the performance issue.

What would you suggest, if I go with the approach of copying first 10,000 records and emptying the DataSet, then again copy 10,000 records and emptying, so on so forth.

To me, the above doesnt seem to be a good idea. I posted the code over msdn forums, codeguru forums, but no replies yet! Sigh | :sigh:

How can I populate the treeview from the SQL server by using the ID and Parent ID of the node?
Confused | :confused:

-- modified at 4:07 Saturday 23rd June, 2007

Joseph Thachil
+91 9916025724

I suspect you'd have to iterate over the data, I don't think a tree view is a data bound control. If it doesn't have a data source property, you need to do it 'by hand'.

Christian Graus - Microsoft MVP - C++

"I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )

I can add parent node and child node, but i can't add child of child node.

+91 9916025724
Joseph Thachil

Its possible to add child of child nodes.

TreeView TreeView1=new TreeView();
foreach (TreeNode tn in TreeView1)
{
TreeNode tn1;

if (TreeView1.Nodes[intTreeCount].Value == treeNodeValue)
{
tn.Nodes.Add(tn1);

}

int childcount=tn1.ChildNodes.Count;
if(childcount>0)
{
tn1.childNodes.Add(tn2);
}

and so on....

I was born dumb!!
Laugh | :laugh:

Programming made me laugh Laugh | :laugh:

!!!
--sid--

i have made simple calculator in csharp but unable to make windows calculator.what should i do.please help
thanks in advance

seeng wrote:
what should i do

Buy a book on winforms and work through it.

Christian Graus - Microsoft MVP - C++

"I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )

T9 dictionary is used in mobiles.i want to make this dictionary using c sharp.give me some idea how to start
thanks in advance

File\New\Project.

What is a T9 dictionary ? If you need to ask how to write a calculator ( as you did above ), odds are you should buy a book and work through it, so you know enough about C# to understand the answers you will get here.

Christian Graus - Microsoft MVP - C++

"I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )

Check this out: http://www.t9.com/[^]. It shows you how a T9 dictionary works on a mobile phone..

May I ask why? I guess your computer has just like mine a keyboard with a lot of keys so that there is no need for something like T9..? What point am I missing?

However implementing such a dictionary is not difficult. You only need a list with the words that can be completed and then you need to check this list whenever you press a new key, starting from the last space, point, comma...

I know that this answer doesn't tell you a lot but your question also doesn't tell me a lot about the problem Sigh | :sigh:

Just start with it and come back if you have any specific problems!

I have a program which comes with an image library. I just created a dll that has a method, which takes a name and returns an image from the dlls resources. Issue is, the dll gets loaded into memory when the program runs, I just want to pull out images, not have the whole library in memory. What suggestions do you guys have for ways of doing this ? I'd prefer something that scales well, I could end up with a resource file that lives on a DVD and is over a gig, and I still want the images to load quickly.

Christian Graus - Microsoft MVP - C++

"I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )