If a job in Linux System Administration / Operations can teach you one thing its how to keep up with the ever changing landscape that Open Source is. I’ve been working with Linux for 20+ years, and with that comes some, hopefully, wisdom of experience. Linux distributions, and Open Source are divergent in terms of change. The more things change, the more things there are to change.

At some point you will find yourself figuring out how to control what parts of the environment need to change and what parts build a stable platform. How do you enable your revenue generation services to evolve and be agile yet keep the platform its built on stable and highly reliable? How do you focus change?

Its these thoughts that have lead me to doubt my Linux faith. The sign posts have been everywhere, but only seem clear in hindsight. I’m thinking about going FreeBSD cold turkey. Even for my desktop workstations.

Apple’s OS X

Apple’s products have been fantastic development machines and I have more than one and use them every day. Apple purchased NeXT in 1997 and used the technology to revolutionize Apple’s operating system. A large part of what we enjoy in OS X has direct roots to FreeBSD. To this day it is very easy to port code written on or for OS X to FreeBSD. (Kqueue anyone?) Much code that builds our SaaS products and revenue streams are developed in a very BSD like environment.

ZFS and DTRACE

Its been lamented in many places that there are no modern file systems available for Linux. While there are options like Btrfs and even bcachfs, these Linux options have significant short comings. Btrfs’s removal from RHEL is telling. ZFS remains as the only matured copy on write file system that is stable and scalable. Its merits have been widely discussed. If you are not familiar with copy on write file systems that checksum your data and you run data storage operations at scale, you owe yourself some research into ZFS. There are some hard lessons here.

ZFS and DTrace are wonderful gifts from Sun when they created the Open Source branch of Solaris, now know as illumos. Many of us from the Linux world know of the incompatibilities between the CDDL and the GPL and lay much blame at Sun’s choice of licensing for why ZFS has not been accepted into the Linux kernel. Indeed, there are issues here. But the technical issues are more complex. The Linux kernel is a very different place than kernels with BSD roots, and porting code can be a challenge to say the least. If you need the latest OpenZFS features (like ZFS encryption) or fixes, FreeBSD will see them far before ZFS On Linux will.

Being a Linux guy I’ve never had a chance to experiment with DTrace. But its seems that anyone who has sings its praises. SmartOS’s docs just can’t hide what they think of DTrace. More than 10 years later, Linux finally gets some native support for similar tracing. So now these finally exist for us Linux users – but beware! These tools are still very much in their infancy. Not to mention that you can use the power of DTrace on OS X today where you are probably developing and reading this!

Visibility and Monitoring Tools

Readers of this blog might have noticed that I do a lot of visibility and monitoring related work. A good rule of thumb for monitoring solutions is that the monitoring platform should be more stable than the stack it monitors. With monitoring and observability becoming a big data problem storing that data becomes an interesting challenge. Some of the most interesting products in this area have incredible Solaris / BSD roots from which they build on the fabled BSD stability. Not to mention ZFS.

Examples here are Dalmatiner-db which has some history with Project FIFO. My friends over at Circonus build their products on top of OmniOS which is a distribution of Illumos. Granted, with the demise of OmniOS, they have moved over to a CentOS platform. But its ZFS that allows their success.

Other Sign Posts

• The popularity of PFSense for firewall and router applications cannot be easily ignored either. Its completely based on FreeBSD.
• Netflix uses FreeBSD quite heavily in their infrastructure. Do some Google’ing about their Open Connect Appliance.
• There’s been a great shift in the Open Source communities away from GPL style licensing toward more permissive licenses. Go, a very new programming language that I enjoy, uses a BSD style license. Did BSD get this part of the puzzle right years ago?
• Packaging in BSD lore is not something I am super familiar with. Its not a stretch to admit that RPMs and DEBs are both, horribly horrid for packaging software especially in today’s world. They are both very painful. Few people I know give praise to a package management system. But I’ve heard oddly wonderful things about IPS from Solaris’s lineage. FreeBSD’s way of separating maintainership of the OS from package management of the tools and applications installed (Ports) has quite a few advantages.
• The SystemD debate. Personally, I don’t know which side to believe or follow. I’ve been taking the punches as they come. Such as SystemD’s DNS Resolver that re-implements (most of) the DNS protocol over DBus and XML. FreeBSD seems quite the haven for the SystemD unification of the Linux distributions.

Disadvantages

What would be major drawbacks of using FreeBSD on a daily basis? Well, Docker would be the first thing that comes to mind. There have been several attemps at getting Docker running within FreeBSD but there’s nothing that seems well maintained or production quality. What makes Docker such a success isn’t the technology. FreeBSD beat Docker to the punch by about 13 years. But, the workflow and ease of use of Docker is what makes it a part of our every day lives. The best way, probably, would be to run Docker like its done on OS X. In OS X, Docker uses the native virtualization APIs, which are wrappers around bhyve, the FreeBSD hypervisor, and run a small Linux distribution as a VM on OS X. The Docker containers run from within the bhyve VM. A lot of workflows used at my clients and by myself are very Docker centric, so this really is a major road block.

FreeBSD doesn’t have the commercial support that Linux enjoys. Google Chrome isn’t available for FreeBSD including the proprietary binary bits that Chrome wraps up and makes nice. The Open Source Chromium and Firefox are included in the Ports Collection, of course. But no Flash. (Who uses Flash anyway?) Similarly, Netflix may use FreeBSD to distribute content but you can’t watch a show from a FreeBSD workstation. The Widevine binary isn’t available for FreeBSD which is the plugin required for Netflix’s DRM. Also, do you use a proprietary, corporate, enterprise-grade VPN solution for work? That’s most likely also a binary only blob who’s vendor may create some poorly packaged mess for Linux, but not FreeBSD.

What Happens Next

I’m not sure. Bruce (most of my clients) isn’t going to up and switch to FreeBSD and the drop of the hat. But, my consulting company is considering it for some of the reasons listed above. I’m very much thinking about transitioning my workstations that I do my day to day work on to FreeBSD. Perhaps its a tool I should have handy in my tool box.

Although, I’m not stopping my work on Linux which leads my to my concerns. Its clear that my faith in this particular ecosystem has been shaken. The quest for the next shinny thing doesn’t mean it doesn’t stand on the shoulders of giants below. But I’m tired of the perpetual quest for Shiny meaning that we must re-invent and re-solve all of the problems we’ve solved before. Were we to always build on the shoulders of those that came before, I think we would have long surpassed the idea of a composable, building block based, infrastructure.