Password file with over 3000 users.

2007/9/18, Ian :
> Hi All
>
> I have a server which runs with normal passwd/shadow/group/gshadow
> files. There are now over 3000 users and I am beginning to notice a
> performance slowdown and it is about time I looked for something more
> efficient - I like the idea of db or cdb database files.
>
> My requirements:
>
> - Must be pam compatible.
>
> - Most users have MD5 passwords, but some are still crypt passwords. I
> do not have ready access to the original plain text passwords.
>
> - I don't want to mess around with too much, so I must have real "Unix
> accounts" (UID's and home directories) for each user.
>
> - Vanilla Debian "deb" packages.
>
> What alternatives should I consider?

maybe pam-mysql ?

--
Wojciech Ziniewicz
Unix SEX :{look;gawk;find;sed;talk;grep;touch;finger;find;fl
ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje
ct;umount;makeclean; zip;split;done;exit:xargs!!;)}

--

* Ian schrieb am 18.09.07 um 17:19 Uhr:
> Hi All
>
> I have a server which runs with normal passwd/shadow/group/gshadow
> files. There are now over 3000 users and I am beginning to notice a
> performance slowdown and it is about time I looked for something more
> efficient - I like the idea of db or cdb database files.
>
> My requirements:
>
> - Must be pam compatible.
>
> - Most users have MD5 passwords, but some are still crypt passwords. I
> do not have ready access to the original plain text passwords.
>
> - I don't want to mess around with too much, so I must have real "Unix
> accounts" (UID's and home directories) for each user.
>
> - Vanilla Debian "deb" packages.
>
> What alternatives should I consider?

Installing nscd might be worth a try.
I *can* speedup your system.

And you do not need to change anything (IIRC) on your system to make
it work.

-Marc
--
+-O . . . o . . . O . . . o . . . O . . . ___ . . . O . . . o .-+
| Ein Service von Links2Linux.de: / o\ RPMs for SuSE |
| --> PackMan! <-- naeheres unter | __| and others |
| http://packman.links2linux.de/ . . . O \__\ . . . O . . . O . |

--

Hi
Ian schrieb:
> ...
> - Must be pam compatible.
>
> - Most users have MD5 passwords, but some are still crypt passwords. I
> do not have ready access to the original plain text passwords.
>
> - I don't want to mess around with too much, so I must have real "Unix
> accounts" (UID's and home directories) for each user.
>
> - Vanilla Debian "deb" packages.
>
> What alternatives should I consider?
Ever thougt about using LDAP? I do not have an installation that is as
big as yours (only about 800 users), but I have no problems with
performance and the server is not the fastest. home directorys are even
exported with nfs and samba PDC with LDAP and Samba for a M$ Domain) is
doing its job too on this box.

best regards

Bernd

_____________________________________________________________________

--

--On September 18, 2007 5:19:15 PM +0200 Ian wrote:

> Hi All
>
> I have a server which runs with normal passwd/shadow/group/gshadow
> files. There are now over 3000 users and I am beginning to notice a
> performance slowdown and it is about time I looked for something more
> efficient - I like the idea of db or cdb database files.

You've got two issues to solve here. Authentication you actually have more
options than UID/Groups mapping which is controlled by the name service
swith library, libnss. LDAP can solve both using libnss-ldap, modifying
/etc/ldap.conf and /etc/nsswitch.conf and then importing your passwd file
into LDAP. That's actually what we use here, though we modified the
'stock' LDAP configuration to allow a substr index on one of the fields
that normally isn't allowed by the LDAP schema. It was causing the LDAP
servers to thrash. The solution is seamless to the users, the LDAP system
supports crypt or md5 based passwords as normal since the password calls,e
tc, are all handled via the normal pam_unix library via libnss, which is
exactly how it does it anyway. The only difference is where it's obtaining
the users.

There also might be a way (and i haven't looke dinto this) to get Linux to
use a GDB/BDB passwd.db like *BSD's do.

>
> My requirements:
>
> - Must be pam compatible.
>
> - Most users have MD5 passwords, but some are still crypt passwords. I
> do not have ready access to the original plain text passwords.
>
> - I don't want to mess around with too much, so I must have real "Unix
> accounts" (UID's and home directories) for each user.
>
> - Vanilla Debian "deb" packages.
>
> What alternatives should I consider?
>
> Thanks
>
>
> Ian
>
>
> --

On Tue, Sep 18, 2007 at 05:19:15PM +0200, Ian wrote:
> I have a server which runs with normal passwd/shadow/group/gshadow
> files. There are now over 3000 users and I am beginning to notice a
> performance slowdown and it is about time I looked for something more
> efficient - I like the idea of db or cdb database files.
>
> My requirements:
>
> - Must be pam compatible.
>
> - Most users have MD5 passwords, but some are still crypt passwords. I
> do not have ready access to the original plain text passwords.
>
> - I don't want to mess around with too much, so I must have real "Unix
> accounts" (UID's and home directories) for each user.
>
> - Vanilla Debian "deb" packages.
>
> What alternatives should I consider?

if you have the libnss-db package (part of nsswitch) installed, you have
everything you need already.

nsswitch.conf allows you to specify "db" as one of the sources for
passwd, shadow, group, and other files. they are read from a Berkeley
db file in /var/lib/misc (i.e. a quick indexed lookup rather than a
sequential search).

there is also a Makefile in /var/lib/misc for generating the .db versions.

to set up:

1. edit /etc/nsswitch.conf and insert the word "db" into the lines that you
want to use the db module.

e.g. change this:

passwd: compat
group: compat
shadow: compat

to this:

passwd: db compat
group: db compat
shadow: db compat

2. edit /etc/default/libnss-db to tell it which dbs to generate.

3. run "cd /var/lib/misc ; make"

4. set up a cron job (to run, say, every 5 minutes) to run the commands
in step 3 (i.e. "cd /var/lib/misc ; make"). you probably want to
redirect stdout and stderr to /dev/null so that root's mail doesn't get
flooded with make's output (mostly "make: Nothing to be done for `all'."
unless passwd,group, or shadow have changed)

see /usr/share/doc/libnss-db and other related documentation for more info.

NOTE: LDAP is also a good alternative, but a *LOT* more work to set up.
libnss-db is a simple way to speed up what already works by putting the
passwd etc files into hashed database files.

craig

PS: this works. i did this several years ago on one server when the number of
accounts grew to about 5000. there is one small catch - with the cron job
running every 5 minutes, there is a small window of time when the source files
in /etc have been updated but the .db versions haven't been regenerated yet.

the nsswitch.conf file will check both the db and the original source files in
order, so it does not prevent new accounts from logging in. for account
deletions, however, the deleted account will still work until the .db files
are regenerated. similarly, password changes will not take effect
immediately.

actually, it's been years - i can't remember if only the old password
(in /var/lib/misc/shadow.db) works, or if both the old (shadow.db) and
new (/etc/shadow) password will work. either way, that's only until the
cron job runs make again (i.e. at most, up to 5 minutes. or less if you
have cron run make more frequently).

if you have written scripts to assist with account
creation/deletion/changing, you could easily modify them to run "cd
/var/lib/misc ; make" after any change, thus eliminating the delay.

you still want the cron job, though, in case there are other ways for a
password to be changed - shell login by users or poppassd or samba, for
instance.

--
craig sanders

--