Slack's Joys of Linux

Blogger Beta

2006-09-02T21:35:00.000-04:00

I've just been able to convert my blogger account to the Google'd up beta. Its not too different from before, just has liberal Google spice. Although, I'm sure under the hood it is quite different. I look forward to using the tool to graphically alter my blogger template as well. Of course, it did break the feed I had setup to my website, linuxczar.net.

To test out Blogger, I shall moan about how embarrassing it is to be an American at times. I do not agree with this war in Iraq and even our President has finally admitted that it has nothing to do with terrorism. (Jon Stewart had some fun with that clip.) The countries that we have invaded have been left in far worse condition. This week I've read reports on CNN of the increase in violence/deaths of Iraqis and a 60% increase in opium production in Afghanistan. What does it take for our administration to admit that something is wrong and attempt to put matters right?

Oooh...the spell checker is a lot better too.

Current and Yum...Goodnight RHN

2006-08-02T17:30:00.000-04:00

I've been hacking on Current again of late. The Subversion trunk now contains support for multiple channels. (Not implemented as parent/sub channels but the benefits are the same.) Also there is support for Yum repomd and a Yum plugin to handle authentication to the Current server. This requires Yum 2.9.3 or higher.

Forth coming in this development series is a user/admin authentication system and a rework of a web frontend most likely using Turbo Gears.

The real question is what has prompted me to do this work here and now. At work RHN has been deployed to support some 1,300 clients for 3 years now and its turned out to give us very few benefits at the cost of a lot of extra work. Points in particular: There is not a sane way to handle old/inactive registrations which end up getting counted as a used license. To maintain RHN in a redundant manner I need to run an Oracle 9iR2 database which is not one of my areas of expertise. Much less is running an Oracle database in a redundant manner itself. My dependency on Oracle is a big risk, backups or no.

The main benefit we get from RHN is a easy place for system administrators to look to see their group of machines and make sure they are online and getting updates. That is valuable. The rest of RHN, to us, is not. Especially so when Up2date is so many years behind Yum.

What's in the future? Well, it appears to be that RHEL 5 will use Yum and a plugin to interface with RHN. That solves parts of the problem. What about RHEL 4? RHEL 3 would be difficult to get the Current/Yum software to work due to Python 2.2, but Current still supports the RHN protocol.

So, where to go from here? I'm very much in favor of having Open Source management software who's goal is to provide functionality like RHN or ZLM from Novell. Would other folks like to help out with a Current/Yum solution?

Auto Updates

2006-07-19T21:19:00.000-04:00

So I have been informed that I have lost the battle for keeping auto updates turned on. I don't quite think so, but that's what I've been told. A co-worker's workstation crashed a few days back and this morning GTK started doing weird things like not being able to understand PNG images. After verifying the PNG libraries were valid I was not able to figure out what was causing problems and my friend decided just to reinstall his workstation. Its sucky. Mess like that has happened to me too.

I really do not know what might have caused the above although apparently this boils down to our servers shouldn't blindly automatically apply updates. This bothers me no end. I will maintain that the man power required to correct problems caused by bad updates are significantly less than our other options. Its been suggested that new updates should be tested or certified and then pushed out to our clients/servers at a defined time. That is not a statement I can disagree with.

However, with who's man hours do we test new packages from Red Hat's updates? Who documents the testing procedures for each update? How can one person verify that update foo will not screw over a badly configured server? Not to mention completely restructuring RHN and all registered machines to cope with this type of policy.

Folks, automatic updates every night is a good thing. It causes much less pain than other methods. Even more so when only one person supports the linux infrastructure of a large university. Using Red Hat's packages rather than building your own saves time for the same reasons. Let's be sane about this.

SuSE

2006-06-26T15:08:00.000-04:00

In my work as the leader of the Linux initiative at my lovely university I've started to look at other "interesting" distributions. Its always good to keep folks on their toes and to make sure that I'm not missing an important bit of technology I should know about or be using.

I've been reading about SuSE's AutoYast. Somebody's definitly on the XML crack. Look at this mess.

http://forgeftp.novell.com//yast/doc/SL10.1/autoinstall/createprofile.scripts.html

Does that not make your eyes bleed?

Work and Stuff

2006-06-06T22:37:00.000-04:00

Well, I'm glad to say that my job search is over. At least for a while. I have accepted a new position at NC State University to lead the Linux initiative at the university level. Really the same job just with the position that says I can do it now. The group I work with is very professional as well. Really is quite a difference.

In other news it seems to be that I'm not the only one that's been left with a sour taste having to use an RHN Satellite as my management and deployment tool for supported Linux clients. Since this university doesn't pay the contract we find ourselves in a catch 22. The contract is administered at a higher level for multiple universities by folks that don't understand the critical nature of security threats and software updates. They even tell Red Hat that we are used to licenses expiring before they can be renewed again. For me this means that while the license is expired my deployment and management system for 1,300 machines is down. Just in time for the latest round of remote kernel DoS attacks as well. As far as RHN as brought us folks seem to want to drop it like a bad habit if we can.

The scary part is that these thoughts and motions lead me to Current and spending some time on it again. Could Current support the university via RHN and the Yum metadata protocols? Could it track my client usage for RHEL license book keeping?

OpenAFS and Kernel Keyrings

2006-05-05T14:15:00.000-04:00

I've been seemingly wasting my time looking at OpenAFS code. Newer kernels have the sys_call_table in a read only code block which prevents OpenAFS from hooking the set_groups() syscall which prevents PAGs from working properly. The solution here is to migrate OpenAFS to use the kernel keyring system written by David Howells. There has been questions, but no movement or code in the OpenAFS community to make use of the kernel keyrings. So I thought I'd look at things myself.

OpenAFS deals with a cred_t struct that contains credential information. Which is in the form of a pointer to the current process's group_info struct. The pointer is copied in on cred creation and the group_info struct is updated for the current process when the cred structure is "set." Only the afs_setgroups() call gets the groups/creds changed for the process which is called by setpag(). However, anytime OpenAFS needs to know the current PAG is digs it out of the group_info struct. Which makes things not easy to migrate to a new system of storing the PAG.

What looks to be a decent idea is to create a couple functions. First, afs_get_groups_from_key() which would return a fake group_info struct populated with the PAG information from the key's payload. Secondly, afs_set_group_in_key() which would save the current PAG information in the provided group_info struct into the current key's payload. Yeah, the names are a little verbose. This would require changes in setpag(), crref(), and crset() as well as not hooking the sys_call_table and providing the key_type. (The crref() function creates a cred_t struct.)

This looks good as it doesn't require API changes in AFS's userland tools. The kernel manages the keys and no more syscall hooking! I haven't quite figured out how the condition will be handled when a process requests a new PAG but that's fairly minor. The big problem is that the kernel keys aren't meant to be used this way. The kernel can search and read/write the payload data in the keys but it cannot manage them. Creating, managing, setting permissions is all supposed to be done by a userland process. Which makes the whole thing a lot more painful. Finally, this email makes sense.

Bugzilla, Current, StateEngine, Work

2006-04-22T13:52:00.000-04:00

The last few weeks have been very full. Lots of evil plans in the works some which I must post about later. I've done a lot of work to upgrade my Linux web servers that serve http://www.linux.ncsu.edu and various internal custom applications that run on them. Its definitely late in the game, but I have put together a repository for RHEL 4 that has all the requirements for Bugzilla 2.20, MoinMoin, and a few other basic python modules for doing web stuff. Thought I would share.

http://install.linux.ncsu.edu/pub/yum/CLS/CLSTools.EL4/repodata

There have been lots of questions about Current of late and I have been thinking about it a lot as well. Not far behind are thoughts about StateEngine. I wish I had more time to really think and be in the mode to code on my personal projects. However, work and life have kept me pretty busy. Does Current have a future? While Up2date in RHEL will go away the RHN protocol will not. At least, that's my view on the situation. I have seen and read the patches for Yum to allow it to support multiple different types of respositories. That's in Yum CVS HEAD now. So I do think Current has a future. I'd like to get Current to support both the RHN protocol and Yum with the ability to require authentication for both protocols. (Not hard with a Yum plugin now.) I'd also like to get the web interface for Current at least as functional as the RepoView tool for Yum repositories. I wouldn't mind some help there. Depending on what fate has in store I may be able to work on Current much more.

Hunter and I have spoken a lot about StateEngine as well. Hunter has been working on some proof of concept code and some methods to get it to work in a specific way. We both see a lot of value in having the tool know about specific state and be able to pull the relevant files off a preconfigured machine. Store those files and lay them down everytime that state is needed. This implies the use of a repository tree so that the same state can have different files/configuration depending on a default, your department, sub group, lab, or host. (Like SSH keys are normally unique per host not per department.) Thoughts there are coming.

A Personal Note

2006-03-23T21:01:00.000-05:00

On a personal note, I was hit by a drunk driver on the 19th of this month. I wanted to post the pics of the damage to my car for some strange morbidly cool fasination I have. This is what the guy did to me:

http://linuxczar.net/images/car-200603/

New StateEngine Snapshot

2006-03-11T21:19:00.000-05:00

I have posted the first snapshot of the StateEngine code on its website:

http://linuxczar.net/moin/StateEngine

This is still very much in the prototyping stage but I hope it shows off some of the concepts that I'd like to work with. At this point there is a directory of configuration files, one per host, that contain the defined state for that host. Each state is defined by a State Object and the arguments or keyword arguments to instantiate it. These files are read into the StateEngine database on startup and the server attempts to spool the resulting Actions to each client. The client code also has a poll option instead of running its own XMLRPC server.

If folks are interested or have feedback for how StateEngine works to deploy and manage configuration please feel free to send me an email or comment.

Time for a New Job

2006-02-18T22:37:00.000-05:00

The bossman got chewed out in a meeting last week. The administration demanding that we come running to every problem that may happen. No budget, no positions for a help desk team or person. It doesn't matter that other colleges have under staffed IT groups that are 4 to 8 times our man power. Oh yeah, and we don't have infinite resources to back up the administrations multiple gigabytes of email per person.

As budgets have been cut repeatedly at the university things have gotten less fun, that's for sure. However, I can only take so much flack for not providing adequate service when the administration cannot be convinced that resources are required to provide those services. Especially since there is no chance of a promotion or raise. I can see the burning bush. Its time.

So, the Linux Czar is looking for a new job. Preferably in the Raleigh, NC area but willing to talk about other adventures. My Resume (PS) is on my website. My interests are in working with Red Hat Products, centralized management and administration of large numbers of workstations and servers, and disaster recovery planning.

Also, I'm sure only Google will notice that I did do some changing in the layout of my website. Mostly to better handle multiple domains better. So older links may need updating.

Google Fun

2006-01-16T19:53:00.000-05:00

Dear Google,

Please index my new website. http://linuxczar.net/moin/TitleIndex

I've created a new website for myself and the various mess I work on. I decided to make it all a wiki which makes maintaining it almost bearable. However, one draw back to using MoinMoin is poor indexing from search engines like Google. Apparently, there have been concerns of Google taking too much bandwidth and CPU time indexing all the pages in a wiki.

Jabber RPC

2006-02-07T17:48:00.000-05:00

I've always wonderd if using XMLRPC over Jabber wouldn't be the way to go while implementing a systems administration tool like StateEngine. It gives you several bennifits such as presence and dellayed receiving. However, I think there are about 2 people on the internet that actually use it. The examples that exist for JEP-0009 are for unmaintained jabber libraries. (Talking Python, of course.) To make matters more interesting the current jabber libraries for Python seem to only work with certain servers and server versions.

This boils down to Jabberd2 apperently sucks. At least that's where I'm at today. The ejabberd server seems to work well with my hackings.

The latest release of Japperpy, which is unmaintained, has a very basic example of making Jabber-RPC go. However, the script is a bit broken and needed some TLC. (Don't you love learning a new language or protocol with broken examples?) I fixed it up and stuck my version of jrpc.py on my web site. This code makes a Jabber-RPC request, bottles it up, sends it to itself via your jabber server, and decodes the method call request.

Next step is go get this example working with XMPPPY, a supported library. From there we'll see if the Jabber RPC thing is actually worth the time.

Not up2date again...

2006-02-02T22:17:00.000-05:00

I've been growing ever dissatisfied with managing RHEL with up2date of late. It doesn't understand the XML Metadata repository format without actually having Yum installed and its feature set is horribly far behind any other updating program. But I think these limitations are already well known.

I've been working on some Yum plugins for kernel modules and seriously considering taking a look at what it would take to make Yum understand the RHN repository format. For those of you that might have heard similar rumors in the past, yes I am the person that randomly talks about Yum and RHN. This is not my first time trying to make this happen.

However, Yum was never designed to handle multiple repository formats. Much of Yum works directly with the XML metadata objects rather than abstract APIs. Its a difficult problem to tackle with the current state of Yum. If anyone has a bright idea I'd love to hear it.

There are sevral ways to go about integrating RHN support. What has been discussed in the past is creating an abstract MetaSack to contain many PackageSacks so you could have one per repository type. The PackageSack object serves as an interface to your caching layer and a single interface to query the entire package set from all repositories. This requires some major changes to Yum and I feel is not something I am capable of doing without active work from folks that are more involved with the project.

Another way I've looked at integrating RHN support is to be able to transmute information from the RHN protocol into the XML metadata. This idea fails because the RHN protocol doesn't provide enough information to generate the XML metadata. A lot of the information needed to resolve dependencies must be gathered after downloading the RPM header from RHN or using the up2date.solveDependencies() RPC call. This method ends up requiring that all the headers be downloaded before any dependency resolving can happen.

I have looked at directly inserting the additional available packages from RHN directly into the PackageSack. This method requires you to hand the SQLitePackageSack an SQLite database object. I can't abstract that. There is much SQL in the PackageSack code. Which goes back to being able to generate the metadata format from the RHN protocol.

I have also looked at creating a Yum Plugin and trying to go around the metadata. In this case it looks like I would be inserting packages into the TransactionData (tsInfo) object. This method would require that I re-implement much of the code in Yum as I need to add the packages as Available, Install, or Update. So I've got to figure out what packages are to be upgraded with that particular invocation of Yum.

Alas, it looks like no RHN support for Yum for a while. Hopefully the MetaSack implementation wont be too far in coming.

Xen and LVM Snapshots

2006-01-13T18:18:00.000-05:00

I've been setting up a meeting at work to try to introduce some of the newer or most helpful technologies present in Linux that other administrators may not know about. Mostly an introduction to RAID, SELinux, LVM, and Xen with a few other goodies for good measure.

A while ago someone (that wasn't overly cluefull) mentioned that LVM snapshots will work with Xen. Of course, they work just like any other LVM volume would. But the idea was cool. Could you setup a machine, snapshot it, and then create multiple, identical Xen domains? Yeah, not too hard.

Xen did not like the yenta-socket module for driving my pcmcia. I knew that Xen didn't support pcmcia but on module insert the system locked hard. I added a grep command to filter the output of kmodule -d in rc.sysinit as well as disabled the pcmcia service.

The xm config file for starting a guest domain can not have spaces in the "name" variable. If it does you'll get a python traceback with the error "No such file or directory." This is documented on the Fedora wiki as happening if there is not enough physical RAM for your guest. I wonder what else causes this traceback?

Also, there are a few known bugs about udev causing funness. In your snapshots you'll will need to create /dev/console or the kernel will be unable to open its initial console.

Xen With Snapshots

Google

2006-01-09T22:17:00.000-05:00

Had I just Googled I could have saved 2 hours of wasted time today. Instead, I kept trying to fix Fedora. Got to find a way to increase the sanity level at work.

Physical Security

2005-12-30T17:56:00.000-05:00

Been reading about the aspects of physical security. Would like to do some more reading on the subject. There are quite a few short papers out there that talk about aspects of physical security or give you some type of checklist aimed at helping you cover all the bases in your data center. One, entitled Data Center Physical Security Checklist, does the checklist thing quite well. Its very good about confining its scope to physical security and is quite professionally done.

This list has given me some good idea about things to do about my server room and, hopefully, more points to take to the faculty regarding how unacceptable the current condition is.

Just to top off facilities randomly chose yesterday to get the vendor out to do some work on our AC system. I have had a call in about some malfunctions we have been having recently and the work most definitely needed to be done. However, facilities did not give us any notice. "Hi, we are starting work on the AC system, the physics department told us we should talk to you." Good grief! We are now back up after a two day degradation. I didn't quite turn everything off, as the basic building AC will hold us to a point.

The physics folks move into the server room's outer room next week.

The Faculty That Crossed the Line

2005-12-14T17:31:00.000-05:00

I feel that this should be told in the form of an epic story. But I'm too stressed out right now to add such a flare. A day that I've seen coming for a very long time has finally come. Well, come and past. It was just today that I was able to put all the peaces in order and connect all the dots.

We have a server room in the basement of the building I work in. The space used to belong to physics. There is an inner room that is the server room and another room you must go through to get into the server room. From the outter room, the server room is not secure. The inner door can be opened with a coat hanger, there are vents between the two rooms, bad dry wall, etc. Its damn cold because of the AC in the server room, and loud. Also, the physics department still controls the keys to this room and gives far too many folks access to what should be a secure area. We also use the outer room for receiving shippments and as a staging area for upgrades of labs which happens two or three times a year. But, the main point is that if the outer room is physically compromised so is the server room.

The day has come where physics wants to move grad students into this outer room. We have made folks aware of other space in the building 10 feet away from this one, but that is too far away from the research group. I've had the bossman make the point of the aspects of physical security of the server room. No argument was able to sway the physics department. The deans and administration refused to back our claims. The bossman is now having our student helpers clean out the outer room and move stuff into the server room. This is creating air flow problems already. Did I mention a fire hazard? Its very bad practice to clutter a server room.

Yet, the battle has been lost. I have no idea when students will move into that space but that room as been re-aquired by the physics department.

What do I do with a physically compromised server room? What do I do when some physics minion opens the server room being cold and tries to adjust the thermostat? What do I tell the users of the equipment in that room?

Why will the administration not back the efforts of the IT professionals? I have no budget and am unable to aquire the expertise and equipment to do my job at what I would think would be satisfactory. No matter what I say, what I do, what policy I try to create the faculty always turn everything around so everything is the fault of the IT professionals. Even a storage/backup catastrophe from a completely different IT group is blamed on my IT group.

There are many things that piss me off. There are many things that make me think about getting a new job away from the university. There are a lot of things wrong that work hard to correct. I can deal with a lot of political mess. But the faculty have gone across the line. There is no turning back. This situation is completely inexcusable. But, I still have no way to convice PhDs that there is something wrong here.

I have an interview with Red Hat tomorrow.

Quota Tool for Users

2005-12-08T18:05:00.000-05:00

Quota is always painful. But its not as hard a problem as, say, what StateEngine tries to solve. Always raising a user's quota when they hit the invisible wall is not practical. In the Linux world, or at least over here at NCSU you get a dialog box that says you are over quota and brings you into Failsafe mode for your graphical login. Failsafe mode, yeah, that mode where you have a crappy xterm in the bottom of your screen. I have a lot of users that have no clue how to use a Bash prompt.

Why hasn't anyone solved this user interaction problem? Why is there not a generic tool that can pop up letting users examine their files and delete some of them. Some limited file management aspects. I realize that there are many different file systems and different ways for querying about your quota. This can be pretty easily abstracted into modules.

This is one of the biggest non-automated tasks I am bothered by. This should be an easy thing for users to fix themselves.

IT at the University

2005-12-07T23:41:00.000-05:00

Something was suggested to me and its true. There are two types of System Administrators at the University. There are those who understand the importance of the computer systems working together in unity. Where the entire system serves the needs of its users, is secure, and dependable. The second type of administrator is a person that believes they are hired to fix computer problems and make people happy.

In all cases, the individual computer as well as a complex IT infrastructure is meant to serve the people at the organization. However, there are many times when a user must be told "No" because what they request would degrade the performance of the IT infrastructure or degrade the services that IT is required to provide. In many cases the administrator must be aware enough that they need to understand the user's problem and to be able to find a solution that does fit into the IT infrastructure. If you see your job as to make anyone and every happy then you will only provide the solutions dictated by the users rather than understanding and solving the actual problem.

When asked about a particular user, in the context of retrieving a file from the backups, a system administrator was not sure that the user's computer was being backed up at all. He replied that one of the things on the list was a backup solution for the departmental staff. He also asked if we had any suggestions on what he could deploy "other than all going back to a distributed computing solution." Yikes.

Unfortunately, it seems that most of the "system administrators" at the university are of the second type. Unable to use the existing systems that we have used to create and restore backups for years. Unable to think beyond what a user told them to do. Unable to grasp the larger concepts that keep IT working in a sane and manageable way.

Now I get to watch as the second type of computer administrator tears down a well build system that's been proven and well used for years because a user said they didn't want to have quota.

Upon No Other Suggestions

2005-10-15T17:00:00.000-04:00

This week I delt with 2 specific bugs with Red Hat products.

The first is that out of about 1,000 machines I maintain I had about 2% - 3% that experienced some sort of error while applying RHEL 3 Update 6. RPM was somehow interrupted in the middle of the transaction which left the machines with many duplicate RPM packages supposedly installed. For example, some machines had two versions of the pam-devel package installed. The next time up2date was run it experienced multiple dependency errors and could not update the system. Along the lines of each version of pam-devel required the same version of pam. There was only one pam package installed. To make matters more interesting all machines that were affected had the same duplicate packages.

Rather annoying to fix this was. Our solution was to rpm -e --justdb package where package was the higher version of one of the duplicate packages. However, my preferred recommendation was to reinstall the machine.

Problem 2: The Red Hat Network servers expose an XMLRPC API so that one can write scripts to modify machines in RHN or somehow use the information stored there. One of the group names had a "&" character in it. When I listed the groups on the RHN server this character was not properly escaped in the XML stream resulting in XML errors. This has been filed at Bug #170553 in Red Hat's Bugzilla.

Adventures in Web Templating Part II

2005-11-07T20:52:00.000-05:00

After writing the code from my last post I have spent some time tweaking it and running some bench marks. Not anything really exact, but enough to give me some idea about the direction I'm headed in.

This direction ain't it. The code is very, very slow rendering web pages. I took a web page right from my existing web pages running through SnapLook. 1,000 hits took over 2 minutes, 17 seconds. Figuring out that I was not using the C ElementTree implementation I got the time down to 1 minute 58 seconds. Holy crap kind of slow. I haven't read all of Kid's code but I believe that since I am inserting XML markup into the template it has to regenerate an ElementTree and validate the template again. Since I already have an ElementTree I'd like to make it simpler but I don't see how. Way too much XML parsing is going on.

SnapLook renders the same web page with a template that is much more featurefull (compared to stick this bit of XML in this tag) in less than 30 seconds for 1,000 hits.

Adventures in Web Templating

2005-11-03T16:17:00.000-05:00

I have continued to follow my interests in what I call Web Templating somewhat described in earlier posts. The basic idea is that I want to create very simple HTML files that through some magic my web server wraps with a nice CSS, menus, navigation, etc. I want my web pages to have a consistent look and feel. I want web pages to be simple for me to create and even possible for my users to create. The only thing I know that does this is SnapLook.

For reasons mentioned in earlier posts I want to be able to have this functionality with my Python web scripts or applications as well as my web pages. So I decided to finally look at Kid a bit more. Apparently, its all the rage. Having really liked TAL and SimpleTAL I did find Kid to be more TAL-ish than I originally thought. Also, its python API is much saner. Its purely XML based which gives it some very winning features versus SimpleTAL is text based. I've never been a big XML person but I have definitely discovered that XML based and text based each have their own merits. Just depends on what you want to get done.

Along these lines, many of the simple web pages are not XHTML. They have <img> and <br> tags. So Kid choked on my web pages. However, once converted things turned out rather nice with Kid creating HTML 4.01 Strict output of my template and web page data. The down side is that many web pages I am not going to convert. Like the piles of DocBook. I can regenerate my current documentation, but the older, historical stuff is much more painful. In general, making all my "simple web pages" into XHTML snippets removes the simple.

Also, I need to be able to pull out certain things from the simple web pages. Such as any meta, html, title, and body tags and do the right thing with the content there. That means I have to parse the page. Non-XML pages can't be easily parsed. Regular Expressions are not fast, but is parsing the XML? This led me down the ElementTree path and to find ElementTidy. Then there was pulling this together when the modules are designed to work with files rather than strings. But, after much reading, much frustration, a dash of code, and being generally pissed off at non-working tape libraries and hot server rooms I create this proof of concept.


from mod_python import apache

from elementtidy import TidyHTMLTreeBuilder
from xml.parsers.expat import ExpatError
from elementtree import ElementTree

import kid
kid.enable_import()

import template

def handler(req):

   req.content_type = "text/html"

   page = template.Template()
   fd = open(req.filename)

   try:
       tree = ElementTree.parse(fd)
   except IOError, e:
       return apache.HTTP_NOT_FOUND
   except ExpatError, e:
       # Bad XML
       fd.seek(0)
       tree = TidyHTMLTreeBuilder.parse(fd)

   page.html = ElementTree.tostring(tree.getroot())
   ret = page.serialize(output='html-strict')

   req.write(ret)
   fd.close()

   return apache.OK

This is a bit of mod_python code so you'll need the propper Apache configs in an htaccess file or some such that associates a filename extension with this bit of mod_python code. It handles XHTML simple web pages and slightly not so XHTML web pages and wraps them neatly in a Kid template. (Which at this point only has one tag in it with the attribute py:content="XML(html)".) The page is properly parsed so that I can edit it slightly as needed and everything sent to the browser in HTML 4.01.

Is this method any better than SnapLook/PHP? How much slower? Its not fast but I've never bench marked SnapLook either. Would other people on this planet think something like this is useful?

Follow Up Notes

2005-10-31T15:24:00.000-05:00

More as notes to myself I did find in CherryPy's documentation how to run CherryPy based apps "behind" apache. Its not the best that I hoped for. But here are the mod_rewrite rules. Some neat tricks that will handle most, if not all, of my concerns. Even an automated way to start the CherryPy app on the web servers. (Which is would be handy in my web server pool.)

CherryPy is the webserver part of TurboGears from the post below.

Had I No Friends

2005-10-29T15:00:00.000-04:00

Had I no friends I think I could write code that might be useful to myself and other people. But alas, I seem to be forced to listen to other folks tell me how bad my ideas are when they do not understand the needs I have. Or, better yet, are just trying to convince me to use their favorite piece of code.

Being the only person that maintains all of Linux at a single university I have a fairly decently sized web site I must maintain. For the last couple years I have been running SnapLook. Its a web templating system written in PHP. I, or other users, can write very simple web pages and the templating system applies a header, footer, css, menus, and navigation. I don't have to worry about make the web pages I write pretty or getting all the navigation stuff correct.

However, I fear that this code will no longer be maintained. Also, I'm not overly fond of PHP code even if it is well written. I'm a Python person and I have lots of web code written in Python. However, most of them aren't complex enough to be considered a Web Application or they simply aren't Web Applications. (Such as XMLRPC APIs or Kickstart generators.) Finally, when I do have code that generates a web page of some sort it can't be put through the templating system. I want all my web pages to go through the same templating system as much as possible so that they all look alike.

Granted, some of what I maintain does tend toward the web application side of the house. Maybe I should look into TurboGears or some other web application framework. (Although, I'd prefer they run through Apache rather than CherryPy or some other code around Python's single threaded HTTP server.)

In any case, my reason of frustration today is that I want all my web pages to have the same look and feel and proper navigation. One way to step torward that is to re-implement SnapLook with Python. And using a template of the design of my web site with Kid, SimpleTAL, or some other HTML/XML templating engine.

Why does every one I know think this is a horrible idea?

Breaking the Camel's Back

2005-10-08T00:59:00.000-04:00

I'm really frustrated at Red Hat. I'm almost at the point of saying "Screw it!" and going in a completely different direction. Any direction. Its not just one or two things that are bothering me, its not a specific person or group. What bothers me so much is the consistent breakage. There's always something rather important that is completely hosed for 3 to 4 months at a time until the next RHEL Update is released.

For RHEL 4 Update 2 the SELinux policy changed in a way that would not allow OpenAFS to work. Could not mount /afs. I filed a bug and got a very quick response. In fact, the problem had already been fixed. Of course it wont be pushed out until Update 3, but I downloaded the most recent RPMS (no source however) and pushed them out through my RHN Satellite to my clients. Overall, one of the better experiences I've had getting things fixed of late.

However, the more this kinda stuff happens the more frustrated I get. If its not SELinux policy, its up2date not handling obsoletes, kernels that can't serve NFS because of 6 month old bugs, nscd agents that wont expire their caches, RHN Satellite stupidness, missing libraries, or any of a number of things that have prevented me from having a good RHEL 4 rollout.

Okay, so this has been a draft for a few days. I must say that today I have discovered many RHEL 3 Realm Linux clients that have dependency errors keeping them from updating. It looks like that up2date somehow crapped out when working on U 6 leaving many machines with 60 some duplicate packages installed. Like gcc-c++-3.2.3-52 and gcc-c++-3.2.3-53.

Maybe I should keep a list of problems I'm having with Red Hat products. I hate to. Red Hat is one company I want to see succeed and do things right. They have been such a leader in that...until the last couple years. What do I do?