Currently working on the wsu wiki. I have added a new topic entitled Eutester User Guide.  Rather than edit the current page for Eutester, I think adding a new topic done in the same style as Euca2ools User Guide will be more efficient and will avoid getting in the way of other projects using the same page.  A huge thank you to viglesiasce, from whom most of this material is sourced.

Rather than just link to the various blog posts, I decided that a wiki format best serves our class’ purpose, and obviously allows for quick editing.  Note: please feel free to add any useful comments or clarifications.  The purpose of the wiki in my mind is to allow someone not familiar with any of the programs and languages (but with some programming background), to quickly become efficient in their uses.

From the blog sflynn1976 » wsu-cs by sflynn1976 and used with permission of the author. All other rights reserved by the author.

This week was more looking through code and understanding what is going on than anything else. After we learned about the paramiko and boto modules that we needed to run the Python code, a lot of the code made more sense. We all want to have a pretty good grasp on the code before we actually start doing anything. But this coming week I will actually start to document a little and hopefully I’ll have a patch ready by the end of the week. I would like to start at the top and work my way down through the different files if possible. For example, I would like to start documenting the setup.py in the highest directory before moving into the deeper stuff if possible, but it depends on where the rest of the group is at with the code I suppose. We could start dividing it all up my directory and each person could document an entire folder (or at least a few files in each directory if there are more people). This week I will email my group and find out what they think is best. Either way, we should have some of the documentation done by the end of the week.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

This week in class we first discussed updates on what people have been working on over the past couple of weeks. Various groups provided information regarding where they were at with their portion of Eucalyptus. Our group, (Myself, Dan Adams, and Brian Labbe), decided what direction specifically we would like to take our documenting of the architecture of Eucalyptus. We decided to split up the project into three main portions which include: Storage Controller/Elastic Block Storage, S3/Walrus, etc. I will be working on the Storage Controller/Elastic Block Storage part of the documentation. As references I’ll be using the following Amazon docs: http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AmazonEBS.html and http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/concepts.html#storage. This is helpful because I will be able to compare the Eucalyptus storage controller to Amazon’s EBS. I was able to find information regarding Eucalyptus’ storage controller here: http://open.eucalyptus.com/wiki/EucalyptusBlockStorage_v2.0. Using these tools and information I was able to begin to create a document that describes how Eucalyptus’ Block Storage functions.

From the blog nzahid » WSU CS by nzahid and used with permission of the author. All other rights reserved by the author.

This week we really learned about what goes into the eutester code to make things happen. There were several pieces that we could not figure out right away, but we spent this week doing research and we discovered what each missing piece actually was. For example, we had no idea what “paramiko” meant at first, but we discovered that it is an ssh module that allows Python code to be compiled and run remotely. Paramiko needs to be installed in order to run the eutester package. There were a few other small modules that we did not understand at first, but we eventually figured it all out and posted links on the wiki about everything we came across.

Now, rather than linking a Eucalyptus instance to a kernel like we wanted to do last week, we are able to actually run Eucalyptus images and ssh into them, which will save a lot of time. So, in moving forward, the Eucalyptus team decided that it would be very beneficial if we went through the eutester package and documented all of the code. Most of it is undocumented, so it can be tedious to read through, but we have a fairly strong understanding of it now, so we should be able to document it and contribute our changes to the project. This coming week will involve a lot of code reading and documenting, but it should be very helpful in the end.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

Progress is being made with our class’ cluster.  Hopefully we will soon be able to add images to it, which will allow us to progress a bit futher in using the Eutester program.  For my part, I have been trying to figure out why I could not ssh into any of the instances started from a Ecalyptus image.

This was a vexing issue, made more so because others in the class were able to make it work.  I ended up installing Euca2ools onto 3 different computers in order to compare the results I got on each.  The problem came down to not being authorized to establish an ssh connection to an instance I had started.  I tried multiple times deleting and adding keypairs, and combing through the available images, thinking that somehow it was the images fault I could not establish a connection.

I finally had the choice to add an instances public key to a known_hosts file in the .ssh folder (I assumed I was doing everything up to this point correctly), but still received an error and no connection was made.  On a whim, I went into the .ssh folder and deleted the files I found there.   My next attempt was successful.

Contemplating why this would occur, I realized that the IP address used for each session remained the same, no matter if multiple instances had been created, either from the same or different images.  The problem was that ssh was uncomfortable with this and so refused to connect.

I performed an experiment, to see if I could duplicate this error.  I established ssh connections to 2 different instances of 2 different images.  I then terminated those instance and restarted them in reverse order.  This established those instances with different IP addresses.  I then attempted to ssh into the instances and got errors stating that the host keys had changed.  I feel as if this error should have been caught a while ago, but my ignorance of using ssh got in the way.

Fortunately, I can now start an instance at will and can resume becoming familiar with Eutester, with the goal of using and modifying this tool.

From the blog sflynn1976 » wsu-cs by sflynn1976 and used with permission of the author. All other rights reserved by the author.

We had spring break this week at WSU so I tried to squeeze in some Eucalyptus time. This week the Eucalyptus web interface was made available and we were able to sign up for an account on our cloud. This was some pretty exciting progress. I’ve also been, once again, continuing to research and document for the “Architecture Overview” portion of the Eucalyptus documentation.

From the blog nzahid » WSU CS by nzahid and used with permission of the author. All other rights reserved by the author.

This week I’ve been continuing to research and document for the “Architecture Overview” portion of the Eucalyptus documentation project. I also have been browsing and reading posts on http://planet.eucalyptus.com/. I found some great and interesting information on the site about Eucalyptus. I have also been trying out some of the commands that are documented in the Euca2ools documentation on the WiKi. Just another week of familiarizing myself further with Eucalyptus has been helpful.

From the blog nzahid » WSU CS by nzahid and used with permission of the author. All other rights reserved by the author.

Last week, Brian Labbe and I looked at the best computers we could get for the amount we had (~$1000).

This seemed to be the best deal:

http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=1262860&CatId=333

Then, IT said they had a handful of machines.  We scrapped the new hardware idea and waited for delivery.  Unfortunately, Spring break was coming up, and few people were willing to come in over Spring break to set the new machines up.  Dave, Mike, and I stepped up to the plate and came in on Monday and set up the machines to avoid an entire week of delay.  Within a few hours, all of the machines were swapped and ready to go.  We also had lunch at Bushel N’ Peck down the street.  Great food, great price, great service.  I highly recommend trying it!

Meanwhile, I just got back from the conference in DC at the NSF.  I had a lot of fun, it was great to meet everyone and get out-of-town during Spring Break.  It was also an all expense paid trip – so who wouldn’t be excited! It’s back to work at EMC in the morning.

From the blog jforkey » wsu-cs by jforkey and used with permission of the author. All other rights reserved by the author.

When Neo went down, it was difficult to determine what the next course of action would be since we need the cluster to get Eucalyptus running, and we need Eucalyptus running to actually use eutester. But in the meantime, we worked on several other small projects to get ourselves ready.

First of all, we moved all of our information from our PiratePad to the wiki so that others could start using our material more readily than before. We all made sure we understand the Python code fairly thoroughly as well. On the wiki, we posted info about eutester, getting the repository downloaded, some of the tutorials on Eucalyptus and Python, and even a very comprehensive API for the actual Python code in eutester.

Stephen and I set off to work on figuring out a few key concepts from the code that we are unsure about for this week. There are several code modules in eutester that look like they have to do with specific hardware/operating system instructions, but we want to find out what their jobs are, so we are doing the research this week/last week.

Additionally, we set other members of the team to the task of learning to handle Eucalyptus instances and trying to ssh into them, since we were all originally having problems doing this. Basically, we want people to be able to run their own Eucalyptus image and ssh into it so they can test on their own, or at the very least, just start learning the basics.

This leads into our final (and possibly most ambitious) “assignment” for ourselves this week. We would like to create a Linux kernel that is bundled with a working Eucalyptus image so that we can just upload this kernel whenever we want to test anything and we always know that it will be a working version and it will be compatible with our stuff. Another positive to this would be that since the Eucalyptus servers get wiped every 6 hours, we would already have the image of our personal kernel stored on a local machine and we could simple reupload when the servers get wiped. There is still much to be done before we can understand everything well enough to get to that point, but I think we are on the right track and our group has made very good progress so far in understanding everything they can.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.