Category Archives: open source

Final Thoughts about CS401/Eucalyptus

I had a lot of fun working in this class it was definitely a difference experience than almost all my CS courses before hand.   I liked working on a big project my only request for the future is to see more code but, something that was really unavoidable for a beta-test style class that we had.  I know it wasn’t anyone’s fault just too many problems with the cluster to really get into it.  Just because we didn’t get coding done doesn’t mean I didn’t learn quite a bit about how exactly working on a big project was like.  Learning to use git, irc, and being on a list server were pretty new experiences for me, aside from gaming I had never used irc, git seems to be invaluable to know since every employer I’ve talked to seemed excited that I already know it.

 

Main thing I’ll take away from the course is exactly what cloud computing is, all the nomenclature we’ve learned, how to work on large scale projects, how to communicate with a large group of people from different time zones and different backgrounds in programming/computer science.

 

Final presentations today were great, seemed like every group found a lot of different topics to talk about and cover and they all seemed to master their specific topic.

From the blog jamescelona » WSU CS by jamescelona and used with permission of the author. All other rights reserved by the author.

Final Wiki Adjustments

I never got around to putting up this blog post last week but I figure it’s worth posting.

 

After siting down with Chris while he worked his way through our wiki was great, really gave me a chance to see how a wiki flows through another set of eyes.  Luckily we’ve all been doing a great job of working with the wiki adding the needed information and adding it coherently.  Only issues really were fixing formatting adding things like, “don’t forget to use sudo” to the top of commands rather than after.  Only “big” changes I made to the wiki was adding a bit on git and how to install python user a package manager.

 

I enjoyed working with Chris getting his input, I think it was smart to do it on my own first, since I knew a bit on how to get everything running from doing it a few times, then having him go through.  Since I never posted this Karl mentioned that there is a whole field of engineering in something like that, actually testing your code/wiki/idea on an audience and jotting down notes.

From the blog jamescelona » WSU CS by jamescelona and used with permission of the author. All other rights reserved by the author.

Test Run of the Wiki

This week I am going to start from scratch and try to install Eutester+euca2ools from our wiki only, keeping notes along the way to see if we need to update anything to make it more clear.  I imagine from what I’ve been reading that we’ve made quite a large amount of progress over this course and when I had to restart last month it took me maybe an hour? (would be been less if I realized >>> meant that you type python ) but that’s besides the point.  This blog post will serve as a note pad for anything our wiki should make clearer.

 

 

Updated this:

I decided that I would find someone I could use that doesn’t really know much about Eucalyptus and at least knows how to work Linux.  It turned out pretty well, I have some notes jotted down that I can finalize the wiki with to make it as fast as possible.  Everything was pretty well laid out, aside form small things like the note “don’t forget to use sudo” after they entered all the commands.  One thing I need to add is that we never directly say to install boto.

 

Overall this test went pretty well.

From the blog jamescelona » WSU CS by jamescelona and used with permission of the author. All other rights reserved by the author.

CS401 – We Have Liftoff! (and other news)

Took long enough, but we now have a working cloud!!! It’s been a real struggle to learn the proper way to configure and install things to get the whole system to work properly. Applying what I learned from Andy Grimm and taking his advice to install Eucalyptus 3 via packages rather than building from source, I was able to start fresh and have a functioning cloud in a little over an hour. I was filled with joy when a classmate succesfully launched a VM instance and logged into it via SSH.

Too bad that the semester is nearly over now. Oh well. I need to finish up my documentation for how a newbie can create a Eucalyptus cloud. I was really hoping to get into some source code before the semester was over but that is just not going to happen. Even though I haven’t written a line of code this semester, the course has been invaluable. I am now far more comfortable interacting with software developers via IRC. I have learned a good deal about Linux system administration through failing over and over again. I still hate writing documentation, but now I have gotten a little practice at least. 😉

Hopefully I can wrap up my documentation project soon and get working on the final assignments for the course: an ethical analysis of Eucalyptus and a slideshow presentation about the work I did over the semester.

In other news, the WSU programming team (of which I am a part of) got 4th place out of 27 at the competition held at Quinnipiac University! We solved 4 (nearly 5) out of the 6 problems. I wasn’t expecting such a high so it came as a pleasant surprise to see that we got 4th. This was my last year on the team so I’m glad we performed well for my final competition.

Oh, before I forget: after this semester is over… I’m done with undergrad! Which means I need to find work, preferably developing FOSS. Just need to finish my resume. So, who wants to offer me a job? 🙂

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

CS 401 – Quality Time with Andy Grimm

On Thursday, the 19th, Eucalyptus developer Andy Grimm paid WSU a visit. Other students and I ate lunch with him and our department chair Karl Wurst. Our lunch bill is being paid by Red Hat, so even if I never work there and get a pay check from them, I can at least say that I had lunch on them one time. 😉

After lunch, Andy and I spent the next 3 hours trying to resolve the various issues that were keeping our cloud from functioning properly. The good news is that I learned a lot about how Eucalyptus works, networking, and some neat command line tricks. The bad news is that our cloud is still not functioning.

As I pointed out in my last post: networking is hard. It is especially hard when working on the university network whose topology is pretty much unknown to me. The first big mistake that I made when I set up the network for the cloud was split the LAN into 2 subnets. The CLC, SC, and CC were on one subnet, while all the NCs were on their own subnet and the CC acted as a router for them. I had misread some networking documentation and mistakenly thought that having the NCs are on their own subnet would make things easier for later on. Andy set me straight about that and I “flattened” the network out so all of the NCs are now on the same subnet as the other Eucalyptus components. Next, we set up ethernet bridges for virtual networks that will be needed for the virtual machines. That was mostly straightforward but through some slight misconfiguration I encountered Red Hat networking hell involving a renamed `eth0` which Andy resolved for me by messing with udev.

I thought that I had a recent enough kernel installed on all of the machines to avoid the kernel bug that breaks Walrus. Turns out I didn’t. So no matter what we did the cloud would never function until that was resolved. A `yum update kernel` and a reboot took care of that. Then, for some strange reason, PostgreSQL processes wouldn’t start. Andy started them manually and things seemed to work just fine for subsequent reboots.

Despite fixing all of these issues and making a ton of progress in such a short time, we were still unable to run a VM instance. Andy and I ran out of time (he had a train to catch and I had to return to work) but he suggested that I essentially start over once again. This time I will be using the Eucalyptus packages that are available from the Eucalyptus repository.

So, another week goes by without a working cloud… but I learned a lot. Many thanks to Andy Grimm for making me a better system administrator.

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

More Wiki Editing and Python Learning

I added a big index at the top of the glossary page, learned how to actually link to certain part of a web page in a PMwiki, was actually a lot easier than what I figured it was here was where I found it, turns out you just make a simple [[#anchor | text ]] at the particular part where you want to reference you #anchor that part then  use [[#anchor | text]] and you’re done.   It was silly tedious to do but, it does give it a quicker and easier way to access the glossary, also props to Dan Adams, the glossary is looking very nice.  Since I learned how to use links without having to just post the ‘ugly’ url same page as the anchor tutorial I have been fixing the entire glossary page so it will read “read more”, “wikipedia entry” or something along those lines.  With those edits I am also fixing the capitalization of each entry.

EuTester has been great for learning Python, I was long over due to learn it anyways.  I started using Zed A. Shaw’s book (great resource for learning a language for free/very cheap) and then after I knew enough to make better sense then I started looking at all the Python that eutester had.  Commenting has been great for learning because it forces me to look at the code, analyze what I am reading, then form coherent ideas from that. Zed’s book along with the multitude of forums while google searching has really been great.

From the blog jamescelona » WSU CS by jamescelona and used with permission of the author. All other rights reserved by the author.

Make the Wiki as Easy as Possible

I’ve been spending some time trying to clean the wiki but also to make it very user friendly.  Instead of text telling you where to go I am swapping that for actually linking the person reading to the page.  If you’re confused while trying to set up your cloud, make credentials or anything else the last thing you want is to have to do is to navigate though a wiki you’ve never used before.

Aside from this I’ve also been commenting code, which I will get to in a second blog post, I didn’t know how to push to git, probably should have taken the 30 seconds to read how to do it. But I just kind of left the saved files on my computer and figured I would just submit them when I asked how to submit..unfortunately each one of them have already been commented and pushed.  My loss. There is still a bunch of code left to look at.

From the blog jamescelona » WSU CS by jamescelona and used with permission of the author. All other rights reserved by the author.

CS 401 – Networking is Difficult

So I haven’t posted here in awhile. In CS 401 I have been trying unsuccessfully to create a working Eucalyptus cloud. At this point the basic system is in place: The core services are running. Users can sign up for an account on our cloud. Users can upload kernels and OS images. However, when a user tries to launch a virtual machine instance everything falls apart. The instances are forever stuck in the “pending” state.

I was using the “managed” networking mode for Eucalyptus. However, upon testing if the network was “VLAN clean.” I followed the instructions here and alas, the VLAN was not clean. It seems like more of a headache to get the VLAN to function than it is to switch to the “managed-novlan” networking mode.

The “managed-novlan” mode required me to manually configure an ethernet bridge for Eucalyptus to use. I followed the instructions in the Eucalyptus 3.0 Installation Guide and everything seems to work (running `service network restart` shows no errors), but now the Cluster Controller cannot communicate with the Node Controllers. This means that there are no available resources advertised for VMs. So now I cannot even try to start up a VM instance. 🙁

I only have basic networking knowledge so these issues have been a real headache for me. The fine folks on the #eucalyptus IRC channel have been helping me out with everything but everytime we eliminate an issue there is another one to take its place.

In other news, Eucalyptus project member Andy Grimm will be visiting WSU this week. I’m looking forward to talking with him and hopefully I can convince him to give me some on-site assistance with the servers.

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.

Behind on the blogs, sorry for that

Unfortunately I mistakenly deleted my Xubuntu off of my virtual machine instead of Ubuntu, don’t ask me how I managed.  I started over the day before but after I installed Python back, updated boto, paramiko, and all that jazz I get to the point where  thought I was in the clear but two lines down:

eutester = Eutester(credpath=”../credentials”)  – gives me a syntax error “syntax error near unexpected token ‘(‘ and I can’t figure out whats going on.   Been on the IRC for the past few hours, no one has been around to explain this to me.  I tried redoing my credentials but unfortunately that didn’t solve the problem.   Turns out you have to run python before entering any of those commands but now I’m getting file not found, still better than a stupid syntax error.

I figured out my problem, thanks to Coady and testingcloud’s blog.   I ended up getting this:

http://i.imgur.com/Cj6Gp.jpg

I’m glad I got that credentials thing sorted out, now I can start helping out much more.

From the blog jamescelona » WSU CS by jamescelona and used with permission of the author. All other rights reserved by the author.

Tutorial – Installing GNU FM

I recently heard about a neat little site called libre.fm that provides a last.fm like service for listening to and sharing your musical interests. Libre.fm, however, runs on free software, respects user privacy, and includes only freely licensed music. The software that powers libre.fm is called GNU FM. I installed it on my home server just to check it out. This post explains what I had to do to get a working installation. The GNU/Linux distribution that I am using is Debian Squeeze. Hopefully this guide can be of help to someone.

Prerequisites

This guide makes the assumption that you already have a functional web server (Apache, nginx, etc.), PHP 5, and database server (either MySQL or PostgreSQL).

Setup Database

I made a MySQL user specifically for GNU FM

CREATE USER 'gnufm'@'localhost' IDENITIFIED BY 'yourpassword';

Create a database

CREATE DATABASE gnufm;

Download Source

GNU FM uses Git for version control. The source is available from gitorious. Clone the repository with the following command.

git clone git://gitorious.org/foocorp/gnu-fm.git

Install Gnukebox

From the gnu-fm directory, copy the server application, gnukebox, to the web server directory.

cp -r gnukebox/ /var/www/

From your web browser, go to http://yourwebserver/gnukebox/install.php. It’s quite straight forward. Enter the information in for your database server like the database name and the user that you have created.

If the web server cannot write to the directory in which you installed gnukebox, you will see a page that prints out the configuration code that you will need to manually insert into the config.php file in the gnukebox directory.

Install Nixtape

From the gnu-fm directory, copy the web interface, nixtape, to the web server directory.

cp -r nixtape/ /var/www/

From your web browser, go to http://yourwebserver/nixtape/install.php. You will enter the same database details as you did for gnukebox and give your GNU FM server a name.

Once again, if the web server cannot write to the directory in which you installed nixtape, you will see a page that prints out the configuration code that you will need to manually insert into the config.php file in the nixtape directory.

Nixtape requires some extra PHP libraries to be installed along with it. Please note that the version numbers of these libraries may be different depending on how far you in the future you are coming from. Which means that copy/paste’ing the commands I give you will not work, but you’ll probably only have to change the version number in the command to make it work.

Install ADOdb

ADOdb is a database abstraction library for PHP. Installing the library takes just a simple apt-get.

apt-get install php5-adodb libphp-adodb

Install Smarty

Smarty is a PHP templating library that nixtape uses for all web pages. There are packages available in the Debian Squeeze repository, but GNU FM does not work with them. The problem is the result of a slightly different path to the files contained in the Debian package. Since the GNU FM repo has a .tar.gz archive of Smarty within it, we’ll just install that instead.

From the gnu-fm directory, cd to the external_dependencies directory.

Extract Smarty and the Smarty text plugin.

tar zxf Smarty-2.6.22.tar.gz
tar zxf smarty-gettext-1.0b1.tgz

Copy the needed files to the nixtape directory.

mkdir /var/www/nixtape/smarty
cp -r Smarty-2.6.22/libs/ /var/www/nixtape/smarty
cp smarty-gettext-1.0b1/block.t.php /var/www/nixtape/smarty/plugins

Create a templates_c directory for compiled templates and make it writeable by the web server user.

mkdir /var/www/nixtape/themes/gnufm/templates_c
chgrp www-data /var/www/nixtape/themes/gnufm/templates_c
chmod 775 /var/www/nixtape/themes/gnufm/templates_c

Profit!

At this point I was able to have a working GNU FM installation. There are still plenty of things left to do such as add music, add a new theme, etc. Maybe once I get around to doing those things I will make a follow-up tutorial. Anywho, I hope this has been of some use to you folks out there in Internet land. Still having a hard time setting up GNU FM? If so, head on over to #libre.fm on irc.freenode.net and ask the developers for help. Oh, and don’t forget to leave a comment if there’s an error or something you would like me to improve in this tutorial. You can also leave a comment and say what a great guy I am. 🙂

Take it easy.

From the blog David Thompson » WSU CS by davexunit and used with permission of the author. All other rights reserved by the author.