OpenSolaris as a Xen domU

From Sfvlug

# -*- python -*-

if xm_vars.env.get('install'):
    kernel	= "/var/lib/xen/images/osol-0906-unix.amd64"
    ramdisk	= "/var/lib/xen/images/osol-0906-x86.microroot.amd64"
    extra	= "/platform/i86xpv/kernel/amd64/unix " + \
		  "- nowin -B install_media=cdrom"
    disk	= [ "phy:/dev/VolGroup00/osolvm,xvda,w",
		    "file:/home/jeff/OpenSolaris/osol-0906-x86.iso,6:cdrom,r"
		    ]
    on_reboot	= "destroy"
    on_crash	= "destroy"
else:
    kernel	= "/var/lib/xen/images/osol.0906.xpv.unix"
    ramdisk	= "/var/lib/xen/images/osol.0906.xpv.boot_archive"
    extra	= "/platform/i86xpv/kernel/amd64/unix -B " + \
		  "zfs-bootfs=rpool/ROOT/opensolaris,"	   + \
		  "bootpath=/xpvd/xdf@51712:a"
    disk	= [ "phy:/dev/VolGroup00/osolvm,xvda,w" ]
    on_reboot	= "restart"
    on_crash	= "restart"

name		= "osolvm"
uuid		= "280b2556-94c8-49e0-b9c8-1127d57cee9c"
maxmem		= 1024
memory		= 1024
vcpus		= 1
on_poweroff	= "destroy"
vif		= [ "mac=00:16:3E:A8:01:04,bridge=xenbr0" ]

The above is how I got started, installed and running with OpenSolaris 0906 in Xen. As anybody should know, installation is just the beginning. Next I sat down and tried to turn my OpenSolaris installation into a functional member of my LAN.

And just what does being a functional member of my LAN mean? In short it means users are authenticated via LDAP, they NFS mount their HOME directories using the automounter, and the machine participates in IPv6. It also means my own user account needs to be treated as the administrator via some mechanism available, be that using su(1m), sudo(1m), or something else. In the case of OpenSolaris, that something else is called pfexec(1).

LDAP Authentication under OpenSolaris

Unfortunately, and leading to much hair pulling, the Solaris pam_ldap and nss_ldap modules don't use SSL the way we can easily manipulate it using the openssl command line. However, they do use the Netscape Security Services package, so I had to learn how to use the tools in that package. First was to install the package:

pkg install SUNWtlsu

Step two, initialize a new security database with the Netscape tool. This will generate three files, cert8.db, key3.db, and secmod.db. Change the permissions as they need to be world readable.

cd /var/ldap
certutil -N -d `pwd`
chmod 0644 *.db

Step three, assuming a copy of your root CA certificate is in /tmp, import it and optionally, list out the database's contents which should now be only your root CA.

certutil -A -n cacert -i /tmp/cacert.pem -a -t CT -d `pwd`
certutil -L -d `pwd`

Now, here's the part that tripped me up every time I tried this. In spite of the fact the commands say TLS, the Solaris client does not use starttls! It seems to only work using traditional SSL. This means the client does not connect to the plain text TCP port of 389, do a starttls command and upgrade to encryption; instead it must connect to port 636 which is LDAPS. I kept trying to do port 389 with TLS and failed.

ldapclient -v manual -a credentialLevel=proxy \
                     -a proxyDN=cn=Manager,dc=example,dc=com \
                     -a proxyPassword=soopersekrit \
                     -a defaultServerList=ldap:636 \
                     -a defaultSearchBase=dc=example,dc=com \
                     -a authenticationMethod=tls:simple \
                     -a serviceAuthenticationMethod=pam_ldap:tls:simple \
                     -a certificatePath=/var/ldap \
                     -a serviceSearchDescriptor=passwd:ou=People,dc=example,dc=com \
                     -a serviceSearchDescriptor=group:ou=Group,dc=example,dc=com

If you leave in the -v flag for verbose output, you'll see a lot of information fly by, followed by one or more services, possibly even networking, being stopped and started. It may take several seconds before the command returns. One final note, I don't know if the serviceAuthenticationMethod parameter is actually required, but it doesn't seem to hurt my case any.

Before you can authenticate a user using LDAP on OpenSolaris, you need to solve one more chicken and egg style problem. In the server list, I used a DNS name for my LDAP server. However, ldapclient just copied /etc/nsswitch.ldap over /etc/nsswitch.conf, and guess what? The original /etc/nsswitch.ldap does not tell the resolver to use DNS for host names -- it uses LDAP! Right, in order to resolve the name of our LDAP server, we need to connect to our LDAP server, for which we don't know the address. I'm sure that in some peoples' minds, this counts as a bug.

Open up /etc/nsswitch.conf in your editor, and change the hosts: and ipnodes: lines to "files dns mdns," getting rid of LDAP in the entries. Of course, if you actually have host information in your LDAP tree, leave it in. I do not. Also, make sure you are only using LDAP to look up data that you are serving with LDAP. In my case I removed the "ldap" entry from all the databases except passwd: and group:. Save /etc/nsswitch.conf and try to identify a user that's only in the LDAP tree using id username, and getent passwd username.

Configuring the OpenSolaris automounter

Fortunately for me, the Linux automounter, autofs, is modeled closely after Sun's autofs. There was one hiccup that got me for a while, which I'll cover in a moment.

First of all, I am not using NFSv4 or Kerberos on my network. Therefore, I needed to tell OpenSolaris to only attempt to use NFSv3 to make connections. Open up /etc/default/nfs and change the NFS client minimum and maximum versions to 3.

#NFS_CLIENT_VERSMIN=2
NFS_CLIENT_VERSMIN=3
...
#NFS_CLIENT_VERSMAX=4
NFS_CLIENT_VERSMAX=3

Now, when I tested mounting my /home from my Linux server, it succeeded. Next is to configure the automounter. Add the following line to /etc/auto_master:

/home           auto_home       -nosuid,nodevices,sec=sys

This is very similar to Linux's autofs. The first column is the file system location, the second is the automount map file to use, located in /etc, and the third is the mount options to use. Never let users create set-user-id or set-group-id files in their home directories, and never let them create devices there, either. Lastly, here's what bit me initially. Even though I could log in to my user account after I set this up initially, I couldn't list my own home directory, and couldn't modify any of my files. The reason was a combination of two factors. First, my permissions on my home directory are 0701, which means the owner has read, write, and execute permissions, and all others just have execute, which is necessary for Apache to read the contents of public_html inside my home directory. The second factor was something to do with RPC, which I must admit I don't fully understand. For some reason, the RPC calls to read my home directory were being done at the lowest privilege level, anonymously. So every ls command was done by the nobody user, who has no read permission. This is the reason for the sec=sys mount option. It tells the client to send the user ID number as the access identifier. Since I'm using LDAP on both the server and the client, my user ID number is the same on both sides. Ordinarily, sec=sys should be the default. I haven't figured out why that was being ignored.

Or course, some will point out that using user ID numbers as the sole source of authentication in NFS is a bad thing, and those people are right. This is one of the problems NFSv4 was invented to address. Of course, you should never run any RPC based service across the open Internet. But this works just fine for my little network, and for many other closed networks.

Modify the file /etc/auto_home with the following contents:

##+auto_home
*       nfsserver:/home/&

The +auto_home directive tells the client to look up the mount points in NIS, which I'm not using. The * in the left column matches anything, and the & in the right matches whatever was called by the *.

Finally, just enable the services.

svcadm -v enable autofs

Put this all together and now I can ssh in using my regular user name, the automounter will mount just my own home directory under /home, because that's where LDAP says my home is, and sshd will find my public key listed in .ssh/authorised_keys, and I log in. Very clean.

OpenSolaris Roll Based Access Controls (RBAC)

Solaris's RBAC system is designed to limit the privileges of the root user. It can also be used to grant varying degrees of limited privileges to otherwise unprivileged users. When I first installed this node, I created a temporary user to utilise until I got LDAP and autofs working. The OpenSolaris installer granted this user access to the root role. Now that LDAP and autofs are working, I want to use my regular account, and I need that user to be able to become root.

Fortunately, configuring this was quite easy. The privilege grants are listed in the file /etc/user_attr. All I had to do was copy the privileges granted from my temporary user to my regular user.

sed -n '/tmpusr/{s/tmpusr/myaccount/;p}' /etc/user_attr >> /etc/user_attr

No services needed to be restarted. Now my regular user account can use the pfexec command.