Yellow Pages

Contents


Yellow Pages

I really love YP. But i didn't fall in love on the first look, but my love grew with my knowledge, constant use and experimenting.

What's YP

Yellow Pages are a distributed network database for some of the important files in /etc. If you have more than two machines connected by a network (ethernet), then you'll probably need such a thing like YP. It will simplify administration greatly. If you want to add a new user, you do not have to edit all the /etc/passwd files on all machines but only change the /etc/passwd on the YP master server and invoke the "magic words". And immediately, all other machines will accept the new user.

YP was invented by Bill Joy and others at Berkeley (it was part of the Berkeley Software Distribution) and Bill took it to Sun Microsystems later. The people at Sun enhanced YP a lot and gave it its todays appearance. They had to change the name (to NIS: Network Information System) too, because British Telecom sued them. Later, they made a totally new approach and called it NIS+. In this document i'll avoid the name NIS, because it may lead to confusion. And BTW: The program names still start with the letters yp...

When to use YP

As long as you can go with editing the /etc files, you don't need YP. This is true for a single machine, and maybe for two or three, where you have a small number of users. If you have more machines and (very important:) more users which occasionally want to change their passwords (and they really should do that once in a while for security reasons) without your help, a distributed network database like YP makes sense.

If the number of machines approaches 300, the number of users raises to 10,000 and the machines are on several network segments, then YP begins to degrade in performance and starts to make more and more trouble. In this cases, something better (like NIS+) is really needed.

If you have a heterogenous cluster, YP is often the only choice, because it was part of BSD Unix and many workstation vendors included it with their Unix. But there are subtle differences...

The YP programs

YP uses a couple of programs to do it's work. I'll describe them here briefly, because it might be convenient to know them already if you read further.

Setting up YP

As example, i'll use a fictive network of six machines (bilbo, frodo, pippin, merry, dick, sam). The machines should be running ok without YP and know each other (/etc/hosts).

Basics and Planning

First, choose a domain name for the YP. Do not confuse this with the DNS domain name. Some people prefer to use the same domain name for DNS and YP, but this is not necessary. But there is a subtle relationship between them and it may lead to confusion. On the other side, it works and it may simplify some things. But make sure you know the little difference. For me, a YP domain name like "Moria" or "Gandalf" is much more enjoying than "bagsend.hobbingen.au".

Create a file /etc/defaultdomain, that contains the YP domainname. Distribute it to all machines, that will share the YP domain.

	echo "Gollum" >/etc/defaultdomain
	foreach host (bilbo frodo pippin merry dick sam)
		rdist -c /etc/defaultdomain $host
	end

YP Servers

Then choose YP servers. This should be machines which are very reliable and will run all the time. If you have a big server in a closed room, then this beast would be a good choice. For a handful of machines, only one YP server is needed. But if you have a lot of machines, then follow this general rules:

YP Master Server

One of the YP servers will be the YP master server. Use the most important and most reliable machine for that task. Make sure, it does have a (otherwise empty) /var/yp directory that contains the YP Makefile.

Hint: On SunOS 4.1.X, there is a copy of the YP Makefile in /usr/lib/NIS-Makefile.

On the master server (in the example: bilbo), invoke:

	domainname `cat /etc/defaultdomain`
	ypinit -m
		bilbo
		frodo
		Ctrl-D
	reboot
Besides some stupid questions, ypinit asks for the name of the master and slave servers (bilbo and frodo). Wait for the reboot to finish and watch the new daemons to start: ypserv, ypbind, ypxfrd, yppasswdd and ypupdated.

YP Slave Servers

On each slave server (in the example: frodo), invoke:
	domainname `cat /etc/defaultdomain`
	ypinit -s bilbo
	reboot
Wait for the reboot to finish and watch the new daemons to start: ypserv and ypbind.

YP Clients

On each client (in the example: pippin, merry, dick and sam), invoke:
	domainname `cat /etc/defaultdomain`
	ypinit -c
	reboot
Do not wait until the reboot is done, the only new daemon is ypbind (you have seen this earlier already).

More thoughts on the setup

It is possible to work around the reboot, but this is difficult on BSD based machines and boring on SysVish ones. If you're an expert (or running a HA cluster) snoop through the /etc/rc* files to find out the necessary commands.

It's very instructive to glimpse into the /etc/rc* files and see which things happen on which event. For example here you can find out, what is necessary for a particular daemon to become started.

For example i wasn't pleased with the way SunOS decided if a machine was the YP master server. I changed the line to

	if [ -f /var/yp/Makefile ]; then
and live happily whith it since years.

How it works

There are a lot of functions in the C runtime library (for short: libc) which are used to browse the /etc files. This saves a lot of work for a programmer and makes YP possible. YP intercepts these functions and instead of browsing the /etc files it asks a YP server via the network for an answer. Examples for these functions are: gethostbyname, getpwent, getservent and getnetbyaddr. Read their manpages and see what these are good for.

YP uses Remote Procedure Calls (RPC) to query a database on the server. These databases are built using "ndbm" and reside in /var/yp/`domainname`. The following /etc files are usually in the YP databases:

On many implementations, there are additional databases like: and more.

Some Operating Systems even implement a general mechanism for that. Solaris 2 does (hence the /etc/nsswitch.conf) as well as Linux with the new GNU libc. On such systems you can easily add more /etc files to the YP (/etc/printcap might be an example).

On the other hand you can easily remove some /etc files from the YP. Simply take out the name of the file from the "all:" line in the YP Makefile and remove the database file in /var/yp/`domainname`. This is often necessary on a heterogenous network where not all machines like the /etc files of a master with a completely different operating system. /etc/services is prone to this disease. Or you might decide that a file isn't changed often enough to allow for such an overhead. I usually do not include the services, bootparams, aliases and timezone in my YP.

How to use YP

Using YP is quite simple (if it's running): Make any change in the /etc files on the master server and invoke the "magic words":
	cd /var/yp
	make
and voila! The changes are instantly available on all machines.

If YP slave servers are present, the master sends a copy of the changed database files to them.

If a user wants to change the password, he should use "yppasswd". It will automatically invoke the "magic words" to update the YP password database. "yppasswd" is often a link of "/bin/passwd".

YP and DNS

SunOS 4.1.X for example doesn't use DNS to lookup hosts. But DNS should be the only trusted source of such information, if the machine is directly connected to the internet.

To overcome this behaviour, you can plug the DNS resolver library into the dynamically linked libc. But if you have YP, there is a much simpler approach: Edit the YP Makefile and look for the -b option. If enabled, the YP server will do DNS lookups, if he doesn't find a hostname in the hosts database. Simply set up an apropriate /etc/resolv.conf.

Host names ending in the YP domainname will be resolved by YP itself. If you can assure, that the /etc/hosts of the master server is always the same as the DNS database for your DNS domain, you can use your DNS domainname as YP domainname. On the other hand never use a legal DNS domainname (other than your own) as YP domainname or something strange may happen.

Getting rid of YP

If you get an old machine from somewhere, it might be used to have a YP. Or if you plan to take a machine with you to a conference or presentation, it won't find it's YP servers. Such a machine simply won't boot up in multiuser mode unless YP gets deactivated. This is simple. Boot up the machine in single user mode and remove (rename) /etc/defaultdomain and /var/yp:
	mv /etc/defaultdomain /etc/defaultdomain.off
	mv /var/yp /var/yp.off
You now might make other changes (hostname, /etc/hosts, /etc/passwd, /etc/fstab and such) and then try to boot it up in multiuser mode.

Adding and Removing a YP Server

The list of YP servers given at the very beginning (at "ypinit -m") is distributed over YP too. A new slave server has to be entered into this database. Unfortunately the data is only available as "ndbm"-database so we have to convert the database to human readable form, edit it and convert it back. The program "makedbm" does these conversion tasks. We'll now add "pippin" as new slave server in our example:
        cd /var/yp/`domainname`
        makedbm -u ypservers >ypservers
        echo "pippin pippin" >>ypservers
        makedbm ypservers ypservers
        cd /var/yp
        make

Then you continue as described above setting up a YP slave server.

The same procedure applies, if you want to remove a YP slave server. Edit the file "ypservers" and delete the appropriate line. Then create the new database and invoke the "magic words". On the former YP server remove /var/yp/`domainname` and reboot.

Unfortunately this does not work for the YP master server.

Changes to make it run smoothly

YP works best, if all YP servers are ok and the network is fine. Terrible things may happen, if that's not the case. Especially older YP implementations (ala SunOS 4.1.X) cannot dynamically rebind to a new YP server. The default behaviour is: A YP client shouts for a YP server on the net as it boots and binds to the first one who answers. If this machine crashes, the client hangs too. To minimize problems, i've adopted some useful techniques.

Seeking the right YP server

If a YP server boots up, it starts the "ypserv" daemon very early. But since the machine is under heavy load at that moment, some other YP server (sitting idle) might answer faster to "ypbind" than the local "ypserv". That's not good, because server does have all the databases himself so there is no use in asking another machine over the network.

And worse: If two YP servers reboot at the same time (which can happen after a power failure), they can bind to each other. This is a particular evil situation, because if one machine stops later (crash, reboot), not only its clients will hang but the other YP server and all of its clients too! And sometimes this server may not be able to recover even if the first server starts up again.

A dataless or diskless client should find the YP on it's NFS server. The NFS server is already a single point of failure and binding to another machine for the YP would create another one. Single points of failure lead to faults that do not sum up but multiply exponentially!

The solution is simple: There is an option for ypbind which allows to change the YP server later (-ypsetme). This can be used to guarantee, that a YP server always binds to itself. And a diskless client can bind to a YP server which is already a single point of failure (so not to add another one).

At the point, where "ypbind" is started, i add "-ypsetme" and later i set the YP server of choice to something reasonable.

	dname=`domainname`
	if [ "$dname" -a -d /var/yp ]; then
		ypbind -ypsetme;	echo -n ' ypbind'
	fi
	if [ -d /var/yp/`dname` ]; then
		sleep 5
		/usr/etc/yp/ypset `hostname`
	fi
	server=`grep ":.*[ 	][ 	]*/[ 	]" /etc/fstab |\
		sed -e "/^#/d" -e "s/:.*//"`
	if [ "$server" ]; then
        	intr -a rdate $server
		if [ "$dname" -a -d /var/yp ]; then
			/usr/etc/yp/ypset $server
		fi
	fi
Note, that the above is only a crude excerpt of the real thing. It should only give you an idea how it can be done.

Changing IP addresses of a YP server

As long as it is not the YP master server, you can make such a change by redirecting the clients and removing the YP on that slave server completely. Change its IP address then and set up the machine as YP slave server from scratch.

If you have to change the IP address of the YP master server, remove the YP completely on all machines and afterwards set up a new one from scratch. All else won't work.

YP Security

None. Sorry.

Ok, ok, just kidding. If your YP does know a /var/yp/securenets, then use it! This will help a lot.

Other tools for security can help with YP too. You really should install a recent version of the portmapper. Either grab a patch from your vendor or compile Wietse Venema's portmapper.

A paranoid admin should definately run the YP daemons under the control of a wrapper-libc.

Shadow passwords can be used under YP, but the YP server should reject every query but "ypmatch" on the shadow or passwd.adjunct database. Sun's "ypserv" unfortunaltely allows list queries, if they come from a port less than 1024, so every non-trusted host on the network (a dumb PC for example) can get all the (encrypted) passwords.

It might be a good idea to transfer the shadow or passwd.adjunct database encrypted to the slave servers if ypxfrd supports this.

Operating System specific aspects of YP

YP differs slightly from one operating system to another. The operating systems not mentioned have almost plain decendants of 4.3reno BSD's YP.

SunOS 4

Sun's RPC-4.0 and XDR packages are the base of every modern YP. In addition to 4.3BSD, SunOS offers shadow passwords over YP and some security (based on IP addresses: /var/yp/securenets). SunOS implements a fixed access strategy in the libc: If YP is not activated, it browses the /etc files, else it asks the YP. There are two exceptions to this rule: passwd and group (and their shadow adjuct companions) are always read from /etc and if they contain lines starting with a "+", special actions are required. Normally, you'll add a line with a single "+" at the end of these files to activate YP, but the "+" mechanism allows more fancy things.

Solaris 2

Under Solaris 2.0, YP had vanished in favour of NIS+, but came back (in usable form) as NISkit-1.1 for 2.3. Since 2.6, NISkit-1.2 is included in the operating system. Solaris does have a general mechanism to browse /etc files (/etc/nsswitch.conf) and can use various sources for that task. For example the line describing the mechanism for hosts could look as follows:
hosts:	files dns [NOTFOUND=return] nis
That means: Look up a name in /etc/hosts. If unsuccesful, query DNS and if DNS doesn't find it, quit. If DNS gives an error, ask YP.

BSD

YP occured first in BSD Unix. Modern BSD Unices like NetBSD and FreeBSD have adopted many of the SunOS and Solaris enhancements.

Linux

Linux makes use of freely available YP implementations. These offer simillar functionality than Sun's YP and have more security features. Unfortunately this focus on security has killed the support for shadow passwords over YP which makes it pretty useless if you have lots of users and you want some security.

Programs that can make use of YP

Besides the programs shipped with the operating system, there are some other useful programs that can make use of YP:
Copyright 1999-2007 Peter Koch