How avoid a software use much system memory even not being loaded any file?

Thanks for reading my topic.
Here Lubuntu 24.04.3.
I see some softwares when is done an file load will use more system memory being a correct behavior, but when closing all files the software continue using much more memory than when as started for example.
OS + software = 1,08 GB.
The software load file = 2,40 GB
close the file memory goes to = 1,24 GB.
Load another file = 2,70 G
close the file memory goes to = 1,48 GB.
for each new file more memory will be allocated even with the software not having any loaded file.
That strange behavior happen in some softwares and game emulators. RPCS3 start using 1 GB and load a game use 3,70 GB, close the game and then load the same game will use now 4,25 GB and closing RPCS3 without game will use 1,60 GB being much more high than when was started.
That is an issue ? Have any configuration to avoid it ?
Have a nice week.

What you see is Linux’s filesystem (page) cache at work. Free RAM is expensive wasted RAM, so, as long as it isn’t needed for anything else, the kernel will just keep filesystem data in its cache.

You can get a somewhat more accurate reading of how much RAM is in actual use by running free -h:

               total        used        free      shared     buffers       cache   available
Mem:            14Gi       7,1Gi       5,8Gi       244Mi       2,1Mi       1,6Gi       7,4Gi
Swap:           19Gi       2,6Gi        17Gi
Total:          34Gi       9,7Gi        23Gi

That’s an excerpt from my system. Note how there is more RAM available than free RAM; it’s essentially free + cache — not quite but close enough for our intents and purposes.

You can forcibly drop that cache:

sudo sysctl vm/drop_caches=3

That drops all cache; see the upstream docs for more info.

2 Likes

Thanks 
 I will try !

You shouldn’t do that on a regular basis, though. Consider that example purely educational. You’ll just hamper overall system performance when dropping that cache. It’s what’s responsible for Firefox restarting really fast, for instance.

2 Likes

How are you measuring the memory use ? Depending on the program you might be interpreting the numbers in the wrong way. E.g. if you’re using freeon the command line then the number given for ‘free memory’ actually means memory not ever used by anything. Memory used for loading a file will be considered part of the cache which will be cleared out if something else needs the memory. The number you actually should be looking at with ‘free’ is not the number for ‘free memory’ but for ‘available memory’.
If you’re using something like top then you get a summary that’s quite similar to what ‘free’ gives you but you also get memory use per program in a much more granular form, which can be configured (what fields to show and in what units). Reading the manual for the program you use will explain what the numbers mean and what exactly is measured by them.
On the other hand ‘memory leaks’ – programs allocating memory and not properly freeing it when it’s not used anymore – happen, they are a classic type of error. If that’s what you’re seeing, then no amount of configuring will help, you can only hope some developer finds and plugs the leak(s).

“systemd-run” with a temporary scope as a container and memory limits can restrict the application.

Like this:

systemd-run --user --scope -p MemoryMax=512M -p MemoryHigh=512M yourapplication

MemoryMax is the limit for the OOM-killer, MemoryHigh is the limit for garbage collection and swapping .

I think what OP might be looking at is what (GNOME) resources shows, or its equivalent in Lubuntu. And I have the strong suspicion that resources reports the Memory: value of systemctl --user status <systemd-user-scope-of-app.scope>, which includes page cache, so just because that number grows doesn’t mean there is a memory leak.

1 Like

That’s kind of counter-productive given that OP is talking about a power-hungry PlayStation 3 emulator as the program he has problems with.
MemoryMax will run the OOM-killer in the scope, so as soon as you hit or exceed the limit the program dies - not what you want.
‘man 5 systemd.resource-control’ says about MemoryHigh “the processes are heavily slowed down and memory is taken away aggressively in such cases” - again, probably not what you’re looking for when running an emulator for a game system.
These options are probably meant for long running low priority server-processes with a slow leak - something losing a few kb every hour, so after a week or so you automatically kill and restart it and everything is fine.

3 Likes

I think nobody really ever thought of what they are meant for. :grin: I’ve watched some talks by Meta/Facebook staff who essentially say, “don’t use those, we’ve tried”; use MemoryLow and MemoryMin instead; reserve a minimum which the OOM killer and general memory reclaim won’t count.

Also, MemoryMax without MemorySwapMax is probably not doing what one wants, because anything exceeding the limit will be swapped out; so long as there is swap space available the OOM killer won’t step in. Try this:

systemd-run --user -p MemoryMax=8M --scope $SHELL

Take note of the unit name and then run

systemctl --user status <unit name>

You’ll see that the swap counter is non-zero; starting any program will just cause more swapping.

Don’t know what they have tried. The cgroup max-limits are working as aspected. With also setting the SwapMax, the memory and swap usage of the application ist cutted at the limmits, the application would not be terminated by the OOM but slowes down when the limits are reached, tested with gnome-software. Last ps and grep VmSwap are after clicking a while around in gnome-software. After terminating the application memory and swap are freed up.

IMHO ervery thing is working fine.

Nobody said they wouldn’t work; they do and that’s the problem — it gets pretty involved rather quickly doing it that way. Plus, performance can suffer if one gets those limits wrong or doesn’t react to changed conditions; it’s micromanagement gone mad.
The idea behind using the Low/Min equivalents is to protect the working sets of the apps from being reclaimed and just let them run freely otherwise.

IMHO ervery thing is working fine.
[Screenshot]

You also used a lower High than Max; the purpose of my example was to show that the cgroug won’t get killed when only setting MemoryMax, because it will just spill over to swap, which is probably not what people expect and very likely not what they want because of extra churn on SSD devices.

See this talk (jumps directly to the interesting part but watch it in full for the bigger picture), where the same presenter (Chris Down), who had advocated for using MemoryHigh/Max previously shows how they reversed course and turned to Low/Min instead. Essentially it comes down to the same sentiment that free RAM is wasted RAM, for High/Max keep memory free and may actually hurt performance because too tight limits might just be causing an app to thrash on filesystem cache, for instance. In summary, it’s more like “protect your workload from reclaim” rather than “protect the whole system from your workload”.

Hey folks, quick comment from your friendly, local community moderator.

While all these points are very interesting, let’s stay focused on providing clear, explicit help to the original poster.

@peterwhite23 and @mylinde we really would prefer that these types of discussion take place in a separate topic in the Lounge, for example, where you can discuss to your heart’s delight.

Thanks for understanding.

3 Likes

Thanks for keeping us in check, @rubi1200. :innocent:

@aug7744, to determine how much of the memory usage you see is actual anonymous memory, i.e. not backed by named resources like files, you can instruct the kernel to reclaim page (filesystem) cache data from just the app in question.

Identify the cgroup

Since every app should be started in its own systemd (cgroup) scope on modern desktop environments, you first need to know, which cgroup to look at. Run this command in a terminal, adapted to your app; let’s use Firefox as an easily relatable example:

ps -eo comm,cgroup | grep '^firefox'

outputs something like this

firefox         0::/user.slice/user-1000.slice/user@1000.service/app.slice/snap.firefox.firefox-935868c4-3425-481d-ac1c-82556e70d2ea.scope

(do not use this result, because the part after snap.firefox.firefox- will be different for every instance of the app; it’s just a randomly assigned UUID)
The last path component (after the last /) is the scope unit’s name and you should take a snapshot of its current memory usage:

systemctl --user status snap.firefox.firefox-935868c4-3425-481d-ac1c-82556e70d2ea.scope | grep Memory:

Or you can run

systemd-cgtop user.slice/user-1000.slice/user@1000.service/app.slice/snap.firefox.firefox-935868c4-3425-481d-ac1c-82556e70d2ea.scope

(without the leading 0::)

That monitors cgroups in realtime, similar to top.

Reclaim filesystem cache

Then you take the long cgroup path, again without the leading 0::, to go to the corresponding cgrougfs path under /sys/fs/cgroup/:

cd /sys/fs/cgroup/user.slice/user.slice/user-1000.slice/user@1000.service/app.slice/snap.firefox.firefox-935868c4-3425-481d-ac1c-82556e70d2ea.scope

That directory contains a pseudo-file called memory.reclaim. You can write how much memory you want to reclaim to that file:

echo 100G swappiness=0 > memory.reclaim

swappiness is optional, but it’s key here, because a value of 0 instructs the kernel to not reclaim swappable (anonymous) memory and only get back what it can from the filesystem cache. I used an outrageously high value of 100 GiB to make sure that everything that can be reclaimed will be reclaimed; you can ignore the echo: write error: resource temporarily unavailable error — it basically means “I’ve tried as hard as I can but can’t give you back any more than I already have”.

After that, look at the output of systemctl --user status ... again, or see how the “Memory” column of systemd-cgtop has changed; that will be the amount of memory the application in question actually (thinks it) needs. In the Firefox example that should be 400 MiB, or thereabouts.

And now for the real fun part - systemd magic :grin:

A probably lesser known feature of systemd’s configuration system is that one can have common config snippet (drop-in) directories that apply to all unit names starting with a common prefix and a(t least one) dash (-); how fitting that all app scopes are named like app-<name>-<UUID>.:wink: Firefox is a bit of an exception because it’s acutally launched by snapd, which uses a slightly different naming scheme. If you want to limit Firefox’s memory, you can do so by the following commands:

mkdir -p ~/.config/systemd/user/snap.firefox.firefox-.scope.d/
cat > ~/.config/systemd/user/snap.firefox.firefox-.scope.d/Memory.conf <<EOF
[Scope]
MemoryHigh=400M
EOF
systemctl --user daemon-reload

That should take immediate effect. Now every Firefox instance will have a soft limit of 400MiB on its memory allowance. But that will have an awful effect on its overall performance, and I posit what you actually want is to protect that absolute minimum from being taken into account, when The Man Comes Around; err, when the kernel’s memory reclaimer or its somewhat humorless cousin the OOM killer[1] comes around. :winking_face_with_tongue:
So make that:

[Scope]
MemoryLow=400M

That way the kernel has all the freedom to do with the rest of the memory as it sees fit. Lots of brain power has been invested into the memory management subsystem and it’s really sophisticated; let it decide what needs to go and what can stay.


  1. more like systemd-oomd which is even more trigger-happy ↩

2 Likes