Memory Used by applications

Question

0.00/5 (No votes)

See more:

Hi all,

I have an application that gathers data from websites.

This app works for days, and the data gathered are saved in database and a list(the data is saved in a list/datagridview). after few days the gatherd data reaches hundreds of thousands of lines.

Now i want to be sure,

1/is the list(contining all the data) is saved in the RAM?

Althought i am pretty sure it is saved in the RAM, i need confirmation.

If it is the case:

2/Its important to stop filling a list using this approch to avoid "System out of memory exception(and maybe other exceptions)" when the data becaome huge?

Hope you can enumerate some actions to avoid/do when working with apps that consume a lot of Memory.

Thanks for you help.

*i can get rid of the list if i am obliged.

Posted 28-Apr-14 23:03pm

Ziee-M

Add a Solution

2 solutions

Solution 2

The list content is stored in the process virtual memory (that is both physical RAM and disk).
Your repository should be the database, not the list. The list should show only a part of the collected data (e.g. the most recent 100 rows). This way you would avoid the exception and make the GUI more responsive.

Posted 28-Apr-14 23:15pm

CPallini

Comments

Ziee-M 29-Apr-14 5:38am

Hi CPallini, i will do my tests and monitor the memory, i will feedback soon.
Thanks for your help.

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

OriginalGriff · Accepted Answer · 2014-04-28T23:12:00

Solution 1

List<T> collections are storing in RAM, yes - the data you write to the Database is not - that goes to disk so it isn't lost.

So if you keep adding items to the list, then yes, at some point you will run out of memory, and you will get an exception - how long it takes depends on the size of the object, the size of your memory, the free space on your HDD and whether your system is 32bit or 64bit.

If you aren't using the info, then don't store it in the List unnecessarily - Dispose of it and get it out of your system when you are finished with it if there are going to be a lot of them used and the heap manager can re-use the memory for the next one (with a bit of luck)

If you are using the info as a cache, to speed up the "go back to the DB and get it" operation, then you probably want to keep it in memory, but you might consider adding an "expiry date" and purging every day of so to free up memory. Not a simple job though!

Posted 28-Apr-14 23:12pm

OriginalGriff

Comments

Ziee-M 29-Apr-14 5:27am

Hi OriginaGriff, I am binding the list to a grid, kind of an animation to make the user feel the application is still running since the process is very long, if i limit the list to 1000, i guess it will do the job? Now it will take me some time to change the code, but i will let the solution as is until i finish my tests, i will be doing other memory monitoring, and i will feedback.
If you may link me to an article talking about your last paragraph, that would help me a lot.
Thanks for the fast and accurate response.

OriginalGriff 29-Apr-14 6:14am

Don't bind it to a grid - that uses up even more memory - and processing power.
If you are just using the list to update a UI element so the user knows something is happening, just update the grid manually, keep it down to 100 or so elements, and remove the oldest entry before adding the new at the end. That way you get the best of both worlds with minimum input from you! :laugh:

Ziee-M 29-Apr-14 11:49am

Hi, i am using list.Clear() to free memory, is it the correct way? because the app "private bytes" keep increasing while "Bytes in all heaps" are stable.

if List.Clear() works correctly, then i think i have a memory leak somewhere.

I am getting out of the main question's topic here.