I consider myself an amateur system administrator. I know my way around a Linux machine. I know how to use basic command line utilities. I know how to use Vim. I work on servers at my internship. I maintain about 4. Now, here I am in CS 401… with 8 machines that I need to not only install an OS on and configure, but make them talk to each other as a full Eucalyptus cluster. It’s taking far longer to set this up than expected, but I feel accomplished because it took a lot of work to get to where I am today.

Understanding Eucalyptus

I came into this project knowing literally nothing about cloud computing. But from reading the Eucalyptus documentation, I have a vague understanding of what the various components do.

The first component is the Cloud Controller (CLC). The CLC is the gateway for access to a cluster. It basically runs the show. Note that I said clusters, plural. The CLC make high-level decisions and talks to the cluster controllers. Next, is Walrus, a persistent storage system for the users virtual machine images. The data is organized into buckets (get it?). Walrus can be installed on the same server as the CLC. The Cluster Controller (CC) is in charge of managing several Node Controllers (NCs). The CC also handles access to the Storage Controller (SC) which interfaces with different storage methods. The NCs are servers that run a virtual machine hypervisor (such as KVM or Xen).  The hypervisor manages all the virtual machines (VMs) that are running on the server. Put it all together and you have yourself a cloud!

Preparing Servers

My class has 8 machines running CentOS 6.1. One server will be the CLC, running Walrus as well. One server will be the CC/SC. The remaining 6 will be our nodes. If combining tasks on the servers becomes a problem, it shouldn’t be too difficult to move them to other servers, the only downside being that we will have less NCs.

I was confused about which VM hypervisor was going to be installed on the machines. All of the Eucalyptus 2.0 guides were giving instructions to install Xen. However, when I tried to install the Xen packages, they were not found. After some googling, I learned that with CentOS 6, support for Xen was dropped in favor of KVM.

Compiling from Source

I grabbed the latest and greatest code from the Eucalyptus Launchpad page. I found 3 helpful documents that allowed me to successfully build Eucalyptus. one two three. I’ve been documenting the process as I go on the CS 401 Wiki. It took A LOT of reading and some help from the #eucalyptus IRC channel, but I did it! CS 401 is now in possession of some fresh 3.1 binaries.

Installing Eucalyptus

Now that I have Eucalyptus 3.1 built, I need to install it on all of our machines. I have been using rsync to copy the binaries to the other servers. The CLC, CC, SC,  Walrus, and NC require different Eucalyptus services to be running in order for them to function. As of this writing I am currently having some trouble initializing the databases on the NCs.

I’ve made more progress this past week than any other week so far. Hopefully we can have a functional system in a week’s time.

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

This week’s (actually last week now, 2/13) class was all about Git. Git is a wonderful version control system that I already use frequently for my own personal projects. Git is a distributed version control system rather than a centralized version control system like Subversion or CVS. This means that everyone that works in a code repository has a full copy of the source code history. There is often a central storage place for the “official” source code but in theory there is no central hub that is relied upon by others.

What did we learn how to do? Well, Karl Wurst made us some test repositories on the CS git server. This included a few example files that we would edit. My team “cloned” the repository and made the changes, and when some of us tried to “push” the changes back… CONFLICT! Prof. Wurst had designed the exercise in such a way that we would see a merge conflict occur. When a merge conflict happens, git alters the conflicted files by putting in both your version and the conflicts marked by special characters. I simply deleted what was unnecessary to resolve the conflict, and “committed” the changes to my repository. Now I was able to push successfully.

In System Administrator land, Mike Iudiciani and I have been working to put together a small cluster on which we will install the Eucalyptus software. We’ve run into a lot of annoying problems with OS installation and the networking configuration. We were first plagued with bad install media, and then we had trouble getting the machines online. Once we finally had good install CDs, I talked to IT about how to configure our machines network interfaces and voila, we were online. Now begins the process of installing the Eucalyptus software.

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

Alright, things are starting to pick up!

The class voted on which project we would be working on for the semester by ranking the projects by preference. Among the possibilities were Irrlicht, Eucalyptus, Firefox, LibreOffice, and VLC. We kept the top 2 for consideration: Eucalyptus and Irrlicht,

Eucalyptus is cloud computing platform that bears many similarities to Amazon’s proprietary EC2 platform. Cloud computing is a huge industry buzz word lately that means a different thing depending on who you ask, so maybe as this semester goes on I will know to define it properly. The big advantage this project has is that our instuctor, Karl Wurst, knows 2 of the developers and they really want us to work on Eucalyptus and will offer us as much assistance as possible. There is an open source and a commercial version of Eucalyptus. Being the FOSS advocate that I am, it concerns me a bit that certain features are witheld and appear only in the commercial product. Maybe it’s because I have nightmares about Oracle and what they are doing to MySQL with open core. On the other hand, it’s great that a company is releasing a (mostly) fully featured product under the GPL 3 license. To work on this project we will be installing some flavor of Red Hat style OS (likely CentOS 6) onto 8 machines and making a cluster out of them. I have never made a cluster before so my inner system administrator is excited to get it up and running. Our goal will be to get a functional “cloud” environment set up running the latest and greatest Eucalyptus 3.0 code, which has not been released into the wild yet. The class will be writing installation documentation and bug reporting, but I’m keeping my fingers crossed that I will be getting some real commits into the code repository by the end of the semester.

The great Karl Wurst (can I have an A, please?) may let some students work on the Irrlicht project as well.  Irrlicht is a cross platform 3D library written in C++. The Irrlicht developers are interested in creating a test suite for benchmarking purposes, and they want my class to help write it. As far as resume material goes, I think Eucalyptus will be the more fruitful project. On the other hand, I do dabble in game development and have used a similar library before called OGRE, as well as used the OpenGL C API directly, so I am interested in this project as well. I don’t see myself actually being able to work on this, even if we are given the option as my time will surely be focused on Eucalyptus.

This class is making do all of the good things that I have been too lazy to learn how to use effectively on my own, like writing these blog posts and subscribing to listservs. Listservs are a great resource for project discussion that I never found convenient enough to actually use. Time to change that.

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

The class is into its third week already but I have neglected to post about week 2. Better late than never.

The focus of the second week was to become acclimated to Internet Relay Chat. I started using IRC prior to the start of this course so I already knew the essential commands. Our training exercise was to connect to irc.freenode.net, choose a nickname, and join the #teachingopensource channel. Once in the channel, we had to communicate with a partner in the channel (and only in the channel! No talking allowed.) in order to give information that they could use to edit our user page on the Teaching Open Source wiki. It was very entertaining to see the class of ~25 students all trying to talk to one another in the channel. I was pleased that I was able to get several people in the class to use irssi as their IRC client, because it rocks.

Outside of class we had to research 2 FOSS projects that the class could potentially work on for the semester. I researched and wrote about VLC media player and LibreOffice. I made sure to choose applications that mere mortals (people who aren’t necessarily tech saavy) use and could benefit from our work. I also made sure that whatever I chose was cross-platform and not restricted on one OS. Both VLC and LibreOffice are big projects that have a large user base and they had documents specifically targetting and helping new developers. Both projects seem like they would have members of the community that would be willing to assist us and help us get acclimated to the codebase. Doing this research taught me how to check to see if a community has low entry barriers. Some projects have no information about how newcomers can help the project. Other projects want help but not enough documentation to get off the ground quickly.

From the blog Dave Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

This is my first post on this shiny new blog for CS-401!

I have been excited for this course for quite some time. I am a very big (and vocal) advocate of FOSS, but I have had very little experience actually participating in FOSS projects (aside from a couple of bug reports). I have found trying to get into a pre-existing project to be quite daunting. A few months back I took a look at the xbmc source code because I wanted to implement a feature but I did not have the time nor the patience to sift through the massive amount of code. My course goal will be to actually overcome this hurdle and offer serious contributions to a FOSS project.

I’m not much of a blogger. I’m one of those people that make a few posts and then completely stop and abandon the blog. Since blogging is a course requirement, I will be forced to be a good blogger, at least for a few months.

I am looking forward to putting all of my collective software development knowledge to use. I already know how to use version control systems, mainly git, but I am excited that is finally being formally taught in a course. I already use IRC but I’m glad that the entire class will become aquainted with its glory. I have some experience with editing Wikis because I occassionally write documentation at work. Planets are a new-ish term for me. I recognized the term but did not know that they were an aggregation of community blog posts.

Working in groups is not something I usually do. I’m more of the “lone wolf” character, but a cooperative group will accomplish much more than a single person. That said, I will try my best to work effectively in a group. I worked with a team of two others to make Nonagon last year and that worked out very well.

I am pleased with the “Teaching Open Source” book so far. It did a very good job of covering the basics of what open source is and why programmers like us should care about it. I found a few small typos that I should probably submit corrections for at some point.

From the blog Dave Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.