Linux Cached Memory

29 10 2008

In Linux, reading from a disk is very slow compared to accessing real memory. In addition, it is common to read the same part of a disk several times during relatively short periods of time. For example, one might first read an e-mail message, then read the letter into an editor when replying to it, then make the mail program read it again when copying it to a folder. Or, consider how often the command ls might be run on a system with many users. By reading the information from disk only once and then keeping it in memory until no longer needed, one can speed up all but the first read. This is called disk buffering, and the memory used for the purpose is called the buffer cache.

Unlike Windows other operating systems, Linux administers memory the smartest way it can.

Since unused memory is next to worthless, the filesystem takes whatever memory is left and caches it in order to speed up disk access. When the cache fills up, the data that has been unused for the longest time is discarded and the memory thus freed is used for the new data.

Whenever an application needs memory, the kernel makes the cache smaller; You do not need to do anything to make use of the cache, it happens completely automatically.

Freeing memory buffer does not make your programs faster… Actually it makes disk access slower.

BUT if for some reason (kernel debugging for example) you want to force the buffer to be freed, you need to set the drop_caches value to 1:

$ echo 1 > /proc/sys/vm/drop_caches

Issuing this command will release all the cached memory and also will stop collecting I/O buffer blocks. Let’s see an example of the effect:

Under normal situations, most of the memory is already cached by the system. But if we force the system to free the memory, you can see in the graph how the memory is suddenly dropped.

The technical details of how this works are explained on the Linux API


Actions

Information

4 responses

3 02 2009
Neto

Hi, how you have make this graph about memory usage – by day ? I know you used rrdtool for make the graph but I dont know how you colect the input data to rrdtool.

thanks

3 02 2009
pcabrerat

I’m using Munin… Basically you just have to install munin, munin-node and any http server. After installation start the daemon and if any of the default plugins fits to your system, it will start graphing the performance data using dayly (like in the above example), weekly, monthly and yearly intervals . On one of my machines, it recognized 25 services without the need of extra configuration. Take a look at the official documentation.

In the case of custom graphs I use a perl script included in nagiosgraph which processes Nagios data and stores it in a RRDB.

29 08 2009
Fabs

Just out of interest, why do you see the need to bag windows out? I assure you that windows vista and later versions have a better caching system than the one found in linux. Linux basically reads random stuff into the cache, where vista algorithmically decides what is more likely to be accessed. Also, vista actually uses all of your free RAM for the cache, unline linux which will almost always leave a decent chunk of ram to go unused. Despite this, you still claim that “Linux administers memory the smartest way it can.”. I guess linux isn’t very smart if it can’t use spare RAM for caching.

5 10 2009
yao

Firstly – Sorry for my English :(

When I copy the file (more than 50 MB, example 700MB) between partitions (on linux 2.6.31 [reiserfs]) – memory usage increasing very high from 150MB to 990MB of my 1002MB RAM, CPU usage 100% and system is not responsible :(

I noticed that used memory was in cached memory and I executed “echo 1 > /proc/sys/vm/drop_caches” and memory was freed from 990 to 180 good, but:

Earlier (on 2.6.29) this situation not happend everything was fine…
Disk work in DMA mode.

PLEASE HELP.

Leave a comment