The Search for a Proper VM Host

The task that's been consuming my past few weekends (and a good deal of my weeknights) has been exploring various options for running various virtual machines on my desktop PCs (as opposed to my "server", which runs Gentoo Linux without debate or question).  "Why?" you might ask, and for good reason, since I am continually asking myself the same thing...  Even I am not sure that this is really the best route to go, but I'll try to enumerate my goals below which may shed some light on my rationale.

My Goals

• Avoid the pain of dealing with pre-release software and frameworks.  I've always been incredibly interested in bleeding-edge technologies and releases and being an early adopter often has its painful side-effects.
• Avoid bloat.  Keep it "clean and simple." I'm a minimalist and I don't like bloat.  Visual Studio may be the best .NET IDE around, but it's friggin' huge!  I may use it (and my other development tools) a lot, but I don't consider them part of my "core OS/tools".  When I boot up, I want to be up and running as quickly as possible; and when I'm running, I don't want anything other than what I currently need (and that includes stupid OS "features") to be hogging my resources, be it CPU, RAM, or even disk space.  I want to load those those in their own VM and turn them off/on when I need them; having them in their own VM also gives me the freedom to store them wherever I want and even transport them other places (like work)!
• Be "Agile".  The key to agility is being able to do things quickly.  I also think a key part of Agile development practices is being able to not only try something just to see how it turns out, but the implied converse of easily being able to roll back to a clean environment before you tried that thing out.  Applying this principle at the OS level is a very powerful concept.  (This is very closely related to the first bullet-point above)
• Run Linux as much and where ever possible.  This one has no real logic to it - it's completely emotionally-driven.  I love the idea of the power I get running Linux and I'd like to harness that power wherever possible.  The downside to this (as I found out in this process) is that the UI in Linux really sucks;  it's ugly to the point of un-usability.  (Yes, I know function, experience, and style are very different things, but in this case style overrules them all.)

My Stabs at Success

1. VMWare ESX Server.  This looked like the ultimate answer, since it appeared to serve all of my goals above.  Unfortunately, the install for ESX didn't really go very smoothly (not to mention the fact that it was slightly, ah, "questionable" - legally speaking - to begin with...) so I gave up on it quite easily.
2. VMware Server on Ubuntu JeOS.  I downloaded and burned a copy of this at 3 AM Saturday morning, but never installed it.  Why?  'Cause I re-read its overview and saw that it is meant to run ON a VM host, not act as one!  Stupid (actually, time for bed...)!  Luckily I only wasted about 20 minutes and a CD downloading and burning it.
3. VMware Server on Ubuntu Server Edition.  Well, I got everything up and running with minimal overview...  Then I realized I didn't just need to host VMs...  I needed to be able to "consume" them on the same machine!  DOH!  In retrospect, this was an incredibly blatant oversight.
4. VMware Server on Xubuntu.  Now we're on to something!  Linux host with very minimal overhead - I had the whole system up and running, hosting VMs, and running the client with only 600 Mb RAM.  This was awesome, and it met all of my goals.  Problem was - it met them a little too well.  Sure, I wanted a minimal system to host VMs with no bloat, but the fact is, I kinda wanted some of that bloat.  IM, Twitter, and even Microsoft Office are the types of things that I want available all the time and not things I want or need to virtualize.  Sure, there are OSS replacements or workarounds that can handle pretty much all of my needs, but I came to the sad, sad realization that Linux as a desktop just plain sucks.  It's powerful, sure - but it looks like crap, and that makes it just plain dreadful to spend most of your time in... especially in a "minimal" environment like the one I was shooting for.
5. VMWare Player on a trimmed-down Vista.  Luckily, I was trying out all of the previous attempts on an extra HD, not actually overwriting my existing Vista install; an install which, quite honestly, I never intended to completely replace as I do play games on it occasionally.  This is the solution I eventually settled for (at least for now), since it seemed the best compromise of all my goals, including those hidden ones I discovered along my journey.  I'm wondering, though, if this really is the best solution...  For starters - you'll notice the other solutions involve VMWare Server whilst this one uses Player.  That's right - since I'm on a 64-bit machine and stupid Vista doesn't allow non-signed drivers (and for some strange reason stupid VMWare doesn't provide signed drivers for Server), I'm left with the crappy, non-feature-ridden VMWare Player.  Which sucks.  Oh yeah, and did I mention Vista is bloaty!?

The Bloat that is Vista

Even after I spent quite a while picking and choosing which services to kill (practically all of them - including Themes - except the bare system ones) in order to make my Vista install as lean as possible, it still took up 15% of my total 6 GB of RAM!  Here's a few screenshots of my system after it's booted up before I've run anything (except SnagIt and FolderShare):

That is not my idea of "minimal", especially compared to my Xubuntu install!  I don't have a screenshot of the latter, so you'll just have to trust me when I tell you - 617Mb.  A fully-running system (including X & XFCE Window Manager) took 617Mb of RAM... and that was without any real attempts at minifying it any more than the default install!  Now I'm left wondering what a stripped-down install of XP would look like....

So, I'm back to square one - facing the dilemma that I've been tortured with for the past 6 years since I began using Linux: suffer with the unsavory UI and awkward UX that is the Linux Desktop, or deal with the bloat that is Windows.

It really is a no-win situation.

Trials and Tribulations: My Slow Windows Guest on a Gentoo Linux VMWare Host

From the very first time I used it, I loved VMWare.  BUT, I didn't exactly love the price tag, and so I went without it at home for a long time, trying to use substitutes like bochs, qemu, and even CoLinux with varying degrees of success to satisfy my virtualization needs...  But nothing ever hit that sweet spot that VMWare did.  And so it stood for quite a few years, and then VMWare released the free VMWare Player.  Yes!  I had to have the VM's created for me, but now at least I had access to some of that sweet, sweet VMWare goodness.  And then my wishes really were granted when I read that they had released a free Server product!  YEAH!!!  It was a matter of minutes before I had it downloaded, installed, and running my Windows Server 2003 image on my Gentoo GNU/Linux host.  Everything seemed to be going great, except after a while I noticed my Windows guests seemed to be a bit slooower than usual.  I had given them a respectable amount of memory, and there wasn't much else going on on this machine, so I started to wonder what was up.  I had never had these problems with my VMs - even these same exact VMs - under Windows, so I started troubleshooting.  I read all of the manuals, how-to's, and whitepapers, then re-installed everything including recompiling the modules...  still the same problem.  Then I finally Googled it.  It took me a while, but after I sifted through the garbage hits I found this gem, which I subsequently ran on my Linux host:

$ ethtool -k eth0

$ ethtool -K eth0 tso off

Now, I don't really know what those two lines do... and I don't really care.  All I know is after I muttered those magical phrases, my VMs all SPRANG to life!  It looks like the entirety of my perceived slowness was really derived from all the symptoms of a bad network connection.  In fact, in retrospect it should've been easy to diagnose: my biggest frustration was copying files over the network from my host to my guest.  At first I chalked it up to SMB, since surfing the Internet and copying files from my Windows machines seemed to be fine.  But, when I tried to copy from my host, the entire VM seemed to slow down, and a 3 Mb file could take upwards of 20 minutes.  Once I fixed that problem, the performance went back to the VMWare I know and used to - that 3 Mb file now took seconds, and the entire OS as a whole seemed to respond much better. 

I have to confess, I was starting to lose faith in VMWare, and even virtualization in general...  I really was.  But now I'm in love again.  Thank you, VMWare!

Footnote:

I mentioned CoLinux before...  It was (and I assume still is) pretty damn awesome for hosting a Linux environment on my Windows machines, but I often had difficultly justifying the time and processor I was giving to it, and subsequently replaced with Cygwin with is the BEST LINUX CROSSOVER environment ever.  I'm now able to drop directly to a Bash shell on my XP workstation and sed, awk, vi, grep, etc. my way to bliss.  This is really awesome due to the fact that my professional life is dominated by Microsoft - being a .NET developer and all - and using a Linux machine at work will never happen (ok, I won't say "never", but not any time soon...).  Cygwin (and Cygwin/X) has been by far the best way for me to get my Linux fix at work, in public... anywhere!  Even at home, curled up on the couch with my laptop, sipping some hot tea....  ah, Linux...  --Oh, are you still here?