How to realese memory of used by data from the database in memory ?

Question

0.00/5 (No votes)

See more:

Hello everybody. i have a problem
i have a data base with 1M records.
i want to fetch data from data base in memory Every thousand record by ADO.NET and do my operation
then release the memory and fetch next thousand record and again ...
do GC release memory in this condition?

What I have tried:

SqlCommand cmd = new SqlCommand();
            cmd.Connection = con;
            cmd.CommandText = "select * from Base";
            DataSet dataSet = new DataSet();
            SqlDataAdapter adapter = new SqlDataAdapter(cmd);
            adapter.Fill(dataSet, "Record");

Posted 26-Jan-19 1:25am

Member 14119435

Updated 26-Jan-19 1:40am

Add a Solution

Comments

F-ES Sitecore 26-Jan-19 7:42am

.net handles memory management for you, there isn't much you can do to release memory yourself. You can google "gc.collect" to see how to instigate memory collection. If you're going through a lot of data then you'll need to page through it, or use something like a datareader rather than filling a dataset as that will let you process the data in memory as you go through the rows rather than loading all the data into memory at once.

Member 14119435 26-Jan-19 7:58am

thanks my friend.
so when i have a lot of records in database ,for example if i fetch them every thousand record and do my operation and then fetch new records , memory is released from Previous data by gc??

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

OriginalGriff · Accepted Answer · 2019-01-26T01:41:00

Solution 1

See here: Paging Through a Query Result | Microsoft Docs[^]

Posted 26-Jan-19 1:41am

OriginalGriff

Comments

Member 14119435 26-Jan-19 7:52am

thnaks for your advice.
but my problem is how to release memory from data that i used them and now i don't need they anymore.
and so fetch new records from database

OriginalGriff 26-Jan-19 8:04am

You don't have any real direct control over memory usage, other than to manually call in the Garbage collector. There is no "release this memory" instruction in .NET, all you can do is remove all references to used memory and let the GC do it's thing.

And the sample "What I have tried" code you show doesn't page anything: it fetches the entire dataset from SQL in one monolithic operation (using memory on both your PC and the SQL Server PC)

Member 14119435 26-Jan-19 8:12am

in "what i have tried" code i mean to say , i use ado.net to fetch.
" all you can do is remove all references to used memory and let the GC do it's thing"
about this statement , how i remove references to used memory?
Thanks for the time you spend to guide me.

OriginalGriff 26-Jan-19 8:22am

variableIDontWantAnymore = null;

Member 14119435 26-Jan-19 8:34am

thanks my friend.
dataSet.Tables["Record"].Clear();
Can the use of this phrase be appropriate too?

OriginalGriff 26-Jan-19 8:55am

A quick scan of the reference sources says that it probably does do the same thing - it appears to reallocate a new instance in memory, which would render the previous data available to the GC.

Member 14119435 26-Jan-19 9:03am

OK.
Thanks for the time you spend to guide me and your advices my friend