Category Archives: CS401

euca2ools install from source(with dependencies)

So some time during week 4 of the project I had some down time, and I decided to learn how euca2ools worked. Simple command-line tools yet still very powerful as well. However recently we run to some problems with the version mismatches; so here it goes: “how to install everything from source with the latest versions”:

Very easy to install from source:
NOTE: make sure to remove older versions of the software.

Download newest boto: https://github.com/downloads/boto/boto/boto-2.3.0.tar.gz

cd /download/location
tar zxvf boto-2.3.0n.tar.gz
cd boto-2.3.0
sudo python setup.py install

Download newest M2Crypto: http://pypi.python.org/packages/source/M/M2Crypto/M2Crypto-0.21.1.tar.gz

cd /download/location 
tar zxvf M2Crypto-0.21.1.tar.gz 
cd M2Crypto-0.21.1 
sudo python setup.py install

Download euca2ools from: https://github.com/eucalyptus/euca2ools

cd /download/location
cd euca2ools
sudo make

And that should be it.

From the blog WSU CS – Live and Code by dhimitraq and used with permission of the author. All other rights reserved by the author.

EuTester Issue Tracker

Following spring break we discussed the current status of our cluster which appears to be really close to complete. We are now nearing the latter half of the semester and it is time to switch into gear. While I have been attempting to install EuTester and connect to the ECC cloud I have also been checking the EuTester Issue Tracker on occassion to see if there are any new updates.

As of right now there are 5 listed issues and or requests:

  • Define Licensing:

    Need to figure out what license we’re offering Eutester under. I presume GPLv3 to be compatible with the Eucalyptus product itself, but we need to be explicit. Also need to make it clear that it’s copyright Eucalyptus.

  • Create a standard set of debug tools available when tests fail:

    When a eutester test fails, we need to provide a standard set of debugging tools back to the user. This debug info should include as much as is necessary to find the possible cause of the behavior. Currently, the Eutester class provides a very basic framework for this, but we’d like to expand it.

  • Create a standardized Eutester log/output format:

    We need to standardize the way we present info about a running test to a user – essentially, output formatting.

  • Testing capability for Jenkins + EC2 Plugin:

    Would like to look at test cases for Jenkins + EC2 plugin and whether this is possible to script. Jenkins has a CLI client but whether this works with EC2 Plugin is another thing: https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI

  • Goals, purpose, and guidelines for contributionL

    Need to produce better documentation:

    1) Readme file should be bolsterd
    2) Add more sample scripts
    3) Ensure sane defaults (ie requests go to ECC by default)

Hopefully after some more research I will have a working EuTester build and can connect to a cloud in order to further research some of these issues. I am also going to be looking into the Python language with a little more detail, I am not very familiar with Python and learning it will better help me understand how to configure and run the scripts for each test case. There is an IRC meeting next week that discusses the basics of EuTester hosted by Vic Iglesias who is very involved in EuTester. I would love to attend to learn more because I am not very familiar with command line installs and configuration. Hopefully that classroom will give me a better understanding on how and what these arguments are doing rather than just typing text and hoping for the best. As of now I have gotten both boto and paramiko fully installed and have downloaded all of the most recent EuTester tools, I will update after I figure out how to perform the eucaops import in order to actually start the process. I have a feeling it has to do with where the files are located and the folder permissions, but I have not found a lot of documentation on setting these up correctly.

From the blog michaelkenny2 » WSU CS by michaelkenny2 and used with permission of the author. All other rights reserved by the author.

CS401 – Eucalyptus and Amazon

Did some more research this week and found more ways in which Amazon EC2 and Eucalyptus are similar. It looks like amazon’s S3 is pretty close to what Walrus does, and Amazon’s EBS is close to what the storage controller does.

This means that we’ll be able to look at amazon’s documentation to be able to fill in some of the gaps of information in eucalyptus’s documentation, which is good.

From the blog blabbecs401 » CS401 by blabbecs401 and used with permission of the author. All other rights reserved by the author.

Work During Spring Break

Prior to spring break I had been attempting to install EuTester and test its functionality with no success. I know I will not  get to spend much time this week working on it because I will be very busy with family in town, so I decided to update my blog now. After reading a few blog posts and looking at some documentation I have come to realize that EuTester has quite a few requirements that have to be met before use.

In order for EuTester to work you must first have boto and paramiko (can be installed through sudo apt-get). Having these utilities provides the necessary tools for quickly implementing an automated test strategy according to the Eutester Basics Part I blog.

Currently I have installed paramiko but have not configured boto to this point, I will update after I have finished with the install of both utilities and can start with the EuTester build.

From the blog michaelkenny2 » WSU CS by michaelkenny2 and used with permission of the author. All other rights reserved by the author.

Researching EuTester

Over the past few weeks we have been discussing whether or not we should break into different groups to focus our individual efforts on smaller parts of the Eucalyptus project. I have joined the grouping working with EuTester, basically EuTester from what I have read is simply a way to test a Eucalyptus or Amazon cloud and it can be pretty much customizable. I have learned that it is written in python code and with a little knowledge of the code you can write specific test cases for it to simulate. After further research I arrived at the github Eutester site located here.

My specific portion of work for this project focuses on the EuTester bug tracker, so I was able to look through the github site and find the issue reports that currently exist. Now at this point in time it really doesn’t help me since I do not have a working EuTester build or access to a cloud to test it on, but that is what I will be working on in the upcoming weeks.

I have just now begun reading the Eutester Basics Part I blog post which explains getting setup in a little more detail, and have also begun using our classes wiki to update my findings and also to read what other classmates are encountering as they try setting up EuTester.

From the blog michaelkenny2 » WSU CS by michaelkenny2 and used with permission of the author. All other rights reserved by the author.

Week 7: Crossroads

Stroke of bad luck this week, we lost the head node due to a failed CPU fan. Dave has had a few bumps in the road due to a combination of software issues and system limitations. The conversation has been to bouncing back and forth solutions to the problem at hand.

Do we revert to CentOs 5 a no longer updated distro in an effort to get the cluster up? I wanted to pass this information to the rest of the class so I emailed this to the whole class in an effort to bring everyone into the loop and to understand the current predicament we are in.

New Hardware

Pros:

Immediately fixes the issue using the kvm hypervisor.
Will be a reusable asset for the CS department for years to come
Less frequent hardware issues

Cons: 

No spare parts or support can be provided by on site techs.
Most expensive option.
Will delay the completion of the cluster.

Old Hardware:

Pros:

Least expensive.
Cluster would be up for class utilization faster.
Our experience could help Eucalyptus streamline instructions for “ancient” machines.
On site tech support.

Cons:

This cluster would most likely only be viable for this semester.
Minimal resources will reduce the flexibility of the cluster.
Higher chance of hardware failure (already been proven)
Requires CentOs 5 which is no longer an updated distro of this OS.
Eucalyptus may not even be interested in new documentation on a build using CentOs 5. (guessing)

My Vote: (New Hardware)

In my opinion after evaluating the factors above, new machines are the better way to go even if it does mean a longer delay on the cluster’s completion. This may not have covered all the pro’s and con’s but I thought this was a good way to show the bottom line issues we are facing.

From the blog Mike's Blog. [ CS 401 ] by Michael Iudiciani and used with permission of the author. All other rights reserved by the author.

CS401 –

Well, we’re into actually working on the project now. I am working on creating some documentation on the general architecture of a Eucalyptus cloud. I feel like this will be a good thing for  the project to have. It appears as though the architecture of a Eucalyptus cloud is very similar to that of Amazon’s EC2. That should be a good starting point for describing the Eucalyptus cloud.

From the blog blabbecs401 » CS401 by blabbecs401 and used with permission of the author. All other rights reserved by the author.

Connecting to an ECC Cloud

After finally familiarizing myself with the Eucalyptus documentation I decided to attempt to connect and establish a VM on the ECC cloud. In order to successfully use EuTester and become familiar with testing clouds I need to be able to actually connect to a working cloud.

This process was a bit daunting at first when I started receiving errors immediately from the start. After much research and a little figuring out of my own I was able to begin connecting to an instance. The problem I had initially was that the directory I had created did not contain the .zip file with my credentials from the ECC site. After putting this .zip file in the same directory and properly extracting them I was able to connect to an instance.

Image

However, the next problem would come shortly after. After connecting to the instance I authorized the security groups and allocated the IPs according to the documentation on the Eucalyptus site. This is where the problems started as I attempted to SSH into a VM instance. I keep receiving connection timeout errors as follows:

Image

I will keep researching this issue and speak to a few people from class tomorrow to see if anyone has experienced the same problem.

From the blog michaelkenny2 » WSU CS by michaelkenny2 and used with permission of the author. All other rights reserved by the author.

CS401 FLOSS Doomed to fail test

Personally, I think that the test we used ( http://www.theopensourceway.org/book/The_Open_Source_Way-How_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL.html ) is pretty silly. The criteria range from mundane to inane, and seem to automatically discount any project that might happen to be windows only (Say, perhaps an open source C#.NET wrapper for DirectX? http://slimdx.org ). I think there is some good information, but other parts of it are strange (Being vaporware means your project is doomed to failure? Who would have guessed?)

In any case, testing eucalyptus with this doesn’t really tell you anything about the project because it’s a long established project.

This week we also split into new groups and started working on seperate projects. I’ll be working on adding documentation to provide a more in depth overview of the eucalyptus architecture.

From the blog blabbecs401 » CS401 by blabbecs401 and used with permission of the author. All other rights reserved by the author.

Exploring Eucalyptus

We are finally beginning to delve into the world of cloud computing. The cloud that is being built on campus is almost complete and the team working on the installation is doing a great job and putting forth  a lot of their personal time to finish setting it up. In the meantime we have been given research assignments to coincide with the Eucalyptus installation to learn more about the project itself. I personally have been researching the bug tracking system that they are using. Currently you can access the bug tracker right from the http://open.eucalyptus.com/ page located under the participate menu item.

Image

From here we are given links to different areas in the Eucalyptus project. You can go straight to the issue tracker and view the current support issues they are experiencing. There is also a link to the launchpad which hosts all of the open source code for Eucalyptus. The launchpad also has a link on the top to view current bugs and lists them by their priority.

I have only just begun researching this portion of the project and the rest of the class is also researching other components. We are doing this to familiarize ourself with the many various aspects of the project and it is helping us learn everything involved in a shorter amount of time. Hopefully by the time we have our next class the cluster will be up and running and we can really begin to learn how the project works and maybe even contribute to it.


From the blog michaelkenny2 » WSU CS by michaelkenny2 and used with permission of the author. All other rights reserved by the author.