Solaris 10: tips and tricks for system administration

I started slowly building up my knowledge of ZFS, and have decided to start with the most tasty things. One of them is a RAID-Z approach.

Jeff Bonwick - a ZFS author and one of newly created OpenSolaris ZFS community leaders, wrote a wonderful article on RAID-Z where he thoroughly explains what's new and cool about RAID-Z compared to RAID-5. Please read this masterpiece here.

In short, here's what's cool about RAID-Z:

1) A traditional weakness of parity-based RAID schemes - write hole - is fixed

2) There's quite a performance improvement because now we have dynamic length stripes - any small amount of data will be treated as a data block of a full-stripe size, and therefore it will be immediately and fully written onto disks.

3) Data integrity improved - in addition to traditional XORs to validate data based on parity knowledge, each RAID-Z block is validated with a 256-bit checksum.

4) ZFS has a self-healing functionality built-in. If one of data disks returns corrupted data, ZFS restores the correct data with parity, compares checksums to validate the result, returns this correct data to the disk operation which requested it, and fixes corrupted data on the disk which gave us the corrupted data. Must be really useful!

That's all for now. Soon there will be new ZFS presentations - and I hope they will have even more information on RAID-Z.

Just like I was going to, I've upgraded my blog engine to the latest version of s9y today, and also did a re-design of it.

MOST changes are already made, but since s9y sources are quite dynamic, there are still few entry points where I have to make changes to make sure both blog languages work properly. Because of this, English part of my blog will have a Russian interface for some time, but entries will definitely be working properly and shown in English :)

Please accept my apologies, I will try and fix this in the next few days.

I've been asked yesterday if it's known yet about the date when ZFS is expected to make its way into Solaris 10. There is no exact answer, but the official ZFS FAQ page hosten at OpenSolaris.org is and will probably be the most reliable source of such information.

This is the answer currently found in ZFS FAQ:

ZFS should be available in Solaris 10 Update 2, which is scheduled to ship in Q2CY06. All such plans are tentative, however, and subject to change.

I just couldn't wait any longer, and decided to try something with ZFS. And what do you know? I've found a very useful advice from Ben Rockwood, which allows us use regular files created with mkfile as virtual disks for ZFS. VERY useful, especially when you really want to play with such a great technology and maybe try various configurations, but there are no spare physical disks for such experiments.

I will, of course, read and understand all the inner workings of ZFS later, but the first impression of it is this - too easy to be true! :)

Here's what I did:



After this the newly made filesystem automatically became available (it's a partial output of df -k):



Enough for the moment! Now the next challenge will be freeing up one of the two physical disks I've got in my Netra t1, so that I could bring it under ZFS control.

I've done it! Finally I've got some spare time to upgrade my Nevada b26 to OpenSolaris b27! I just can't wait to play with ZFS :)

I've done the upgrade using binaries - so I didn't have to compile anything this way, but instead simply downloaded BFU archives.

For those of you unfamiliar with BFU, it's a super-fast way to update your system. Naturally, it's less reliable than other standard ways to upgrade your system. But this is because it's much faster - a standard BFU image will be around 60-200Mb, which is nothing compared to the 3-4 ISO images of installation CDs you'll end up downloading from Sun if you want to have a standard upgrade reinstall. BFU archive is so small because essentially it is a set of system binaries with accompanying scripts. Only the necessary stuff. No packages - instead everything is split into archives of general purpose (generic.kernel, generic.lib, generic.root, generic.sbin, generic.usr) an a hardware platform-specific archives - for instance, SUNW,Netra-T12.root and
SUNW,Netra-T12.usr.

If you want to know more - I urge you to read a wonderful Open Solaris Developer's Guide in full.

Now, here's how I've upgraded my system:

1) Went to the "Sun Download Center" from this page - http://www.opensolaris.org/os/downloads/ and downloaded two files:
opensolaris-bfu-20051116.sparc.tar.bz2 - that's the BFU archive itself - all the abovementioned files are here - your new kernel, necessary modules, new commands, etc for this new OpenSolaris build
SUNWonbld-20051116.sparc.tar.bz2 - a set of commands for ON building and installation - I only needed few files. In particular, I needed /opt/onbld/bin/bfu script which does the actual update.

2) Unpacked both archives and installed the SUNWonbld package. The first archive created an archives-20051116 directory for me.

3) Set up the environment variables required by /opt/onbld/bin/bfu script:


Added /opt/onbld/bin to my super-user PATH.

4) Started the actual update, specifying the FULL PATH to my BFU archive:
/opt/onbld/bin/bfu /export/dist/bfu/archives-20051116/sparc

If everything goes as it should, your system will spend quite some time updating itself - you'll see a list of updated sections and a list of conflicts will be shown as well.

You'll have an output of this installation in /tmp/bfu.out* file.

5) Now it's time to resolve our conflicts. Usually these are configuration files which already exist in your Solaris installation, but there are newer versions of them in your BFU archive. All the new versions you'll find in the /bfu.conflicts directory on your system, and the full list of conflicting files will be in the /bfu.conflicts/NEW file.

You'd better resovle all the conflicts BEFORE rebooting - your system may simply never come up if you bump into a particularly nasty conflict. Because of this, do read through the list of files and decide for yourself, how critical they are to you.

But before doing this - there's a better option. Simply start the /opt/onbld/bin/acr command, and it will analyze all your conflicting files and decide which ones are indeed critical for you. After your run this command, it will create a log file in /tmp directory, which will look like this (output is abridged):



So, if you have a zero returned after processing a particular file - it's nothing dangerous. I've had only zeroes everywhere, so I did reboot right away and held my breath, awaiting for my server to at least start responding to my pings. Luckily, everything went okay and my server came up shortly after this.

Well, that's all I have for the moment. Unfortunately, I didn't save my terminal history, so I couldn't demonstrate EVERY step of BFUing. But the next time I'm upgrading - I'll be sure to get back to this entry and update it with more info.

Good luck with your OpenSolaris upgrades!