Linux Czar

A friend of mine mentioned a book that’s been making the rounds a LISA and other groups of skilled sysadmins.  The Practice of System and Network Administration.  So I looked it up on Amazon and ended up ordering myself a copy.  I must admit, I’m very impressed with what I’ve found so far.  The book is full of those rare gems of wisdom that sysadmins collect over the years.  In what I’ve read it really seems to do justice to passing on knowledge of the soft side of being a systems administrator.

The first chapter has several lists of things every sysadmin should have or know, or points to follow to assess and survive specific situations.  One of the points that I found quite useful was in dealing with getting your tools back after you’ve loaned them out.  The most common thing, of course, is loaning out a screw driver.  The suggestion of this book was to keep several screw driver sets handy and just give that person their own set rather than loan out your own tools to be lost.  Don’t accept the set back either.

This really struck home because I spent all of what I think of as my junior sysadmin years with a $10 Black and Decker screw driver kit.  Its a great kit with 80% of what you need to have for cheap.  I normally see them on display in Home Depot quite a bit.  I’ve never seen a ratcheting screw driver kit that would do better for the cash.

So, that begs the question: What do I use for a screw driver today?  My choice is the Husky Pro ratcheting screw driver.  It comes with very few (although good quality) bits, but I’ve collected my own set of bits.  (By some extra philup #2 bits while your at Home Depot.)  The screw driver itself feels really comfortable and easy to use.  The ratchet controls wont accidentally flip to another setting.  It has a long shaft not often found on a ratcheting driver that makes it easy to work with screws inside a dense rack of equipment and wires.  It is magnetic which is handy, and contrary to common belief will not harm computer componets unless you are working with a floppy disk or tape cartridge.  When I just need a normal phillups #2 I also have a few drivers from Klein Tools.

Friday night I had the privilege to meet Maddog thanks to my good friend Drew.  Yet another night on the streets of Boston. And quite the learning experience.

Here are Nalin and Nitin!

Build systems suck.  Mostly because they are a lot harder that you would think to put together.  Its just been recently that we have the Koji system from Fedora and the Open SuSE build system.  Implementing those on site for your customized or possibly closed environments can be very non-trivial.  In my case, I found that RHN, being my gateway to RHEL packages, made using Koji very difficult.  Especially as I wanted to be able to trigger a rebuild of kernel modules every time a new kernel was available.

So, I made a poor man’s build system for my own stuff at work.  I took a lot of ideas from Fedora’s CVS based package system of a year or two ago and created something similar based on Subversion.  Toss in Mock and you have the bare essentials of a build system.  Those being 1) Backups of all your package work, 2) Versioning of your package work and each EVR released 2) Clean, repeatable builds in a controlled build root.

I’ve posted some documentation on the CLS Wiki at work which should be enough for anyone to look at my SVN repo for an example and to clone the magic for your site.  Note that, you’ll need to change the Yum configurations stored in common/mock for the Mock configs as outside folks wont have access to my yum repositories of RHEL.

• https://secure.linux.ncsu.edu/moin/RealmLinux/CLSPackages

Today I was able to track down some weird dependency circles that where happening on a very few number of my machines.  Turns out my fedorakmod plugin was trying to install kmods for kernels that the installonlyn magic had on the erase list.  So I’ve added code to check for kernels being removed to fedorakmod.  Speaking of, the new upstream for fedorakmod is here:

• http://linuxczar.net/git/?p=fedorakmod.git;a=summary

That’s also now in git head for yum-utils.

Continuing to poke around I discovered that the postresolve hook in yum that the fedorakmod plugin uses comes before the installonlyn magic.  I also patched that so the postresolve hook comes directly after.

Almost a successful day.

Well, I’ve touched up and included my article on RHEL Installation Numbers while working through the new website.  You should see it on the right.  I’ve added a small bit about the mysterious Red Hat Global Desktop product as its now included in the installation number code.

I do admit that I was quite livid when writing that stuff up originally.  I edited out quite a bit as my personal feelings on the matter aren’t really appropriate.  Professionally, I found them to be a giant pain to integrate into a deployment of over 1,000 linux workstations and servers.  One of the many joys of Open Source is the ease that different products can integrate and the INs made things much more complex.

I do nasty things to RPM packages, but I feel as if I should point out some great examples of things YOU SHOULD NEVER DO.  Such as:

Version: %(echo `awk '{ print $1 }' %{SOURCE1}`)
#Release: %(echo `awk '{ print $2 }' %{SOURCE1}`)%{?dist}
Release: %{expand: %(awk '{ print $2 }' %{SOURCE1})

Working on updating LinuxCzar.net.  Never fear, old content will return just in a new format.

Some time this week NC State University will make an official announcement that a new email and calendaring system has been chosen that will unite the two different email and calendaring systems used on campus. In a 4 to 3 vote Novell GroupWise will be recommended as our one communications tool to rule them all. NCSU will transition to the new systems over the summer in what will cost upwards of 1 million dollars.

Its amazing that an “Open Source University” that has been charged with reducing IT budgets and pressed strongly to be more efficient with resources will not even consider an Open Source solution for email and calendaring. Our Open Source, Cyrus based email system for most staff and all students has been a strong model for the success of Open Source, its scalability, and cost effectiveness in both man hours and cost. Its a sad time at NCSU.

I’m bored. There are two things that I’m looking at working on. The first is a long standing quest to turn NCSU’s Linux website into a dynamic site that easier for me and others to maintain. I’ve been poking at making the site completely in the existing MoinMoin wiki or redoing it in MediaWiki. Although there don’t seem to be many tools to convert a Moin wiki into MediaWiki. I’ve also been looking at using WordPress and Drupal. WordPress is inviting as its easy to maintain and seems to put a lot of effort into looking really good. The problem is theming the CMS properly. Unfortunately, CSS and similar magic isn’t really my cup of tea. All solutions will take quite a bit of work.

In a similar vain what value exactly does a separate wiki get me? Is there value in having an “official” website with the tacked on wiki? Should the website be one or the other completely?

The second project is configuration management. I’ve been researching the existing tools such as Bcfg2 and Puppet and have been fairly impressed, but neither seem to fit it what what I need. I have some brief requirements written up here:

• http://www.linux.ncsu.edu/moin/ConfigurationManagement

I keep thinking what I want is something based off a distributed SCM such as Git. Where local admins can rebase the configuration tree to achieve what they might need. Mostly, marking a specific file as “I know I purposely changed this, don’t change it back.”

I’ve also been keeping tabs on Func which is very interesting. It may definitely play a role in my monitoring. But it doesn’t seem to be heading in the CM direction either. Its close…perhaps if I thought enough about how to implement some CM ideas on top. Perhaps after Func can pull as well as push.