Is there a way to free up the real memory that Ruby's process is holding?

I remember Ruby as an implementation that is hard to get free even if the object is discarded.(It was a long time ago, so maybe it's different now?) Also, there are many implementations that call free does not return memory to the OS and reassign it the next time you malloc.

So, generally, we're going to program it on the premise that freeing up memory doesn't reduce the amount of memory we have.

If that's the case, you'll need to develop a fixed combination of Ruby's version (if any) and malloc/free implementation that frees up memory.

TL;DR

Ruby doesn't run out easily after rss increases to the maximum amount of memory used, so let's add more memory or load swaps.

Body

Let's talk about MRI ruby.

Ruby, like any other process, has a heap and stack in addition to a static area, a program area (such as a ruby body or a library of native builds). Increasing rss in ruby's process is mostly due to enlarged heap.

There are two main types of data that the ruby program maintains on the heap:

Here are some of the ways to keep the data on each heap in memory:

In addition, the logic for each memory release is as follows:

And that is, ultimately, what matters is how malloc and free behave.And for glibc malloc/free, which ruby uses by default, the following conditions are required to return free memory to the operating system:

The
• (main) heap can be reduced by cutting the head only when the head memory has been released.

Because of this behavior, ruby (or rather long-term running programs for most dynamically malloc/free systems) has behavior such as keeping memory up to the maximum amount of memory used and not decreasing easily.

If there is a long-running program that uses a lot of memory only once in a while, I personally expect it to be swapped in/out well.

We examined Ruby trends between 1.9 and 2.6 in Ubuntu 18.04 LTS on Windows Subsystem for Linux on Windows 10 Pro 1809 (64bit).Ruby is installed with rbenv-install.

In conclusion, I was able to confirm the decrease in rss for each version.However, the increase/decrease pattern and the amount of memory were different depending on the b version, so I couldn't grasp the trend.

• 1.9.3-p551
  In the first round, the number of mountains increases and decreases, but after the second round, the number of mountains increases and decreases slightly only at the end.
  The final value is greater than the initial value.
• 2.0.0-p648
  A pattern with four mountains is drawn, and the final value is almost the same as the initial value.
  The initial value is greater than 1.9.3-p551, but the maximum or final value is less than 1.9.3-p551.
• 2.1.10
  2.0.0 - Almost identical to p648.
• 2.2.10
  It's almost the same as 2.1.10, but consumption has increased about 1.1 times.
• 2.3.8
  Increase or decrease in approximately the same pattern as 1.9.3-p551.
  Overall, consumption increased by about 1.7 times from 2.2.10.
• 2.4.5
  It is almost the same as 2.3.8, but it decreased by 0.7 times from 2.3.8.It's a little more than 2.2.10.
• 2.5.3
  It is almost the same as 2.4.5, but it has increased 1.3 times from 2.4.5.It's a little less than 2.3.8.
• 2.6.0
  A pattern with four mountains is drawn, and the final value is almost the same as the initial value.
  However, the initial value is as large as the final value of 2.5.3.
  The maximum value is about the same as 2.5.3.
  Same waveform with --jit.

Looking at the release notes of each version, performance improvements around GC and memory are often made, and the way malloc is used and free is also changing.Also, if you enable jemalloc, there were different versions, so I think the ease of fragmentation will have an impact.

Therefore, this time, it seems that the OS did not recover the memory simply in the questioner's environment.See if a larger capacity (in hundreds of MB) really doesn't free up any memory.