Swap configuration for 16GB of RAM, 8 cores

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/31/07 15:18, wrote:
> I am installing two servers, each with 16GB of RAM, two quad-core Xeon
> processors, and a SATA hard drive. The machines will be compute
> servers, meaning lots of concurrently logged in users, each running
> an assortment of jobs, and various long-running processes. The jobs
> will be
> vastly dissimilar with regards to using the memory, disk, and CPU, and
> to
> their run time, so the configuration should be generic, general
> purpose.
>
> What are the current best practices with regards to swap partitions?
> Is it
> better to create one big, or several smaller swap partitions? Is the
> rule
> of thumb still RAM*2 for the total size?
>
> We are running Debian 4.0/Etch with the stock -i686-bigmem kernel.
>
> Any input or pointers will be highly appreciated!

Create swap *files* instead. See "man mkswap" for the technique.
Put them on /.

That way, if you happen to need more swap at a moment's notice, you
can have a GB of swap within 5 minutes.

- --
Ron Johnson, Jr.
Jefferson LA USA

Give a man a fish, and he eats for a day.
Hit him with a fish, and he goes away for good!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGr6dxS9HxQb37XmcRAtdgAJsEv4Q0smQBrDX1iNooE68XqIXvVgCcD46+
pd3a4PNc1MYR6oOWHizeSyA=
=CdRi
-----END PGP SIGNATURE-----

--

agenkin@gmail.com wrote:
> What are the current best practices with regards to swap partitions?
> Is it better to create one big, or several smaller swap partitions?
> Is the rule of thumb still RAM*2 for the total size?

For a server I turn off Linux memory overcommit. This avoids ever
having the out-of-memory killer run and kill random unrelated
processes. Especially in a multiuser environment or in an enterprise
level environment this is very important.

But turning off overcommit means that the rule of RAM*2 comes back
into play.

Search the web for linux kernel out of memory killer and memory
overcommit and you will find much discussion about it. Start here.

http://lwn.net/Articles/104179/

http://linux-mm.org/OverCommitAccounting

http://www.redhat.com/magazine/001nov04/features/vm/

I recommend trying following in order to disable linux kernel memory
overcommit. This will cause malloc() to fail when memory is
exhausted. (The default is that malloc() and fork() will always
pass.) This should avoid the linux kernel out of memory killer from
becoming active and killing random processes on the system. This will
cause only the programs that are asking for more memory to fail. This
is the traditional Unix memory model. I recommend configuring servers
that need reliable operation this way.

sudo sysctl -w vm.overcommit_memory=2

HOWEVER! If you do not have enough virtual memory / swap space at the
time that you activate this configuration then it may cause your
system to be unable to fork() and malloc() starting at that moment.
If the system becomes unusable due to fork failure you may need to
reboot. You must have enough virtual memory configured to handle all
of your expected simultaneously running applications. Be careful that
you can reboot your server remotely if you do not have physical
access.

Since the above 'sysctl' test is a temporary configuration a reboot
will reset to the default allowing memory overcommit and enabling the
out of memory killer if the system runs out of memory and needs a
reboot to be saved. If you decide that is a good configuration to
keep it memory overcommit disabled then it may be configured in
/etc/sysctl.conf to be set to off at boot time. This is what I
recommend when there is enough virtual memory available.

Bob

--

On Tue, Jul 31, 2007 at 08:18:09PM -0000, wrote:
> I am installing two servers, each with 16GB of RAM, two quad-core Xeon
> processors, and a SATA hard drive. The machines will be compute
> servers, meaning lots of concurrently logged in users, each running
> an assortment of jobs, and various long-running processes. The jobs
> will be
> vastly dissimilar with regards to using the memory, disk, and CPU, and
> to
> their run time, so the configuration should be generic, general
> purpose.
>
> What are the current best practices with regards to swap partitions?
> Is it
> better to create one big, or several smaller swap partitions? Is the
> rule
> of thumb still RAM*2 for the total size?
>
> We are running Debian 4.0/Etch with the stock -i686-bigmem kernel.
>

Try this:

Use LVM for everything. Actually, grub may or may not work on LVM so
put a regular partition (32 MB is more than ample) at the start (sda1)
and the remainder of the disk as sda2, as physical device for LVM. Then
set up your usual separate 'partitions' under LVM. To be generous, this
means a / of 512 MB, a /usr of 4 GB, a /var of 6 GB plus whatever other
stuff you need for your /var, a separate /srv if needed of whatever
size, and /home. Your choice of filesystem is another matter; it
should be growable and reliable, shrinking is less of an issue. I use
JFS.

Then put swap on LVM as well. Since you may need to increase swap size,
and I don't know how to do that with the swap online, you may want two
swap partitions so that one can be off line for a couple of minutes.

Total swap size depends on how much VM gets assigned. Since disk space
is cheap you may want to start with RAM*2 so you could start with two 16
GB swap partitions.

Note, however, that I run amd64. I don't know what the swap size limits
are on a Xeon.

In combination with your swap partitions, you could use swapd to
dynamically make swap files. You can monitor their existance and as
needed, take a swap partition off-line (swapd should then automatically
increase swap-file size), increase it with LVM, mkswap again on that
partition, then swapon that partition.

Good luck,

Doug.

--

Quoting "agenkin@gmail.com" :

> I am installing two servers, each with 16GB of RAM, two quad-core Xeon
> processors, and a SATA hard drive. The machines will be compute
> servers, meaning lots of concurrently logged in users, each running
> an assortment of jobs, and various long-running processes. The jobs
> will be
> vastly dissimilar with regards to using the memory, disk, and CPU, and
> to
> their run time, so the configuration should be generic, general
> purpose.
>
> What are the current best practices with regards to swap partitions?
> Is it
> better to create one big, or several smaller swap partitions? Is the
> rule
> of thumb still RAM*2 for the total size?
>
> We are running Debian 4.0/Etch with the stock -i686-bigmem kernel.
>

Awesome, you have tons of kick ass electronic hardware.
16 Gigs of ram.... Dualy Quad Xeons...
and only one hard drive? LOL
With that much hard core gear, I wonder if swap at RAM*1 or RAM*2 or
RAM*.5 would make much of difference?
I would be interested to know if it does, if you ever find out whether
one out performs another.
Have you considered using the AMD64 port?, or perhaps you have some
custom 32 bit stuff that can only run on x86?

Mike