This week I spent most of my time figuring out what was left to finish in our eutester repository. I also used a lot of screenshots from github for my powerpoint presentation. I worked with Nate, Coady, and Marcos to put together our final project. We all put together screenshots and code samples from the changes we added to the repo to show everything we accomplished this semester.

Also, I wrote my paper on the Eucalyptus Ethical Analysis this week. It was interesting to look at the software and find out where it was good and where it was not so good. As it turns out, Eucalyptus is not 100% free and open source. It is better described as “open-core” because it only allows certain aspects of the product to be used for free. To add on more powerful features, users must pay a fee, which goes against the entire open source mindset. Users are only given a free trial of the software and all of the products that can be used with Eucalyptus are proprietary. There are other options that are similar to Eucalyptus, that are actually free and open source, but it appears that Eucalyptus is more mainstream and user friendly. 

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

This week I tried to download a fresh repository and none of my credentials worked. I have been unable to access the new repo to make further changes, so I’m not sure if I will be able to make anymore pushes by the end of the semester. My group has managed to get a decent amount of work done, so I am very excited about that. However, it is unlikely that we will be able to finish the entire project with only a week remaining in the semester and none of us  able to access the new repository.

Overall, it has been fun and interesting learning how to use the Eucalyptus software and writing code and documentation for the eutester package. I wish we could have done more, especially on the development side, but it was a large project and it was difficult to actually get started doing anything. It took the entire semester to get any semblance of a cluster set up, and I still do not believe it is in a working state, not to mention the fact that more than half the class had to learn Python before being able to interpret any of the code. I really wish we were able to develop some real code as a class for the project, or at least finish the project we were working on. There were simply too many variables for an overly ambitious project that contributed to the overall lack of contribution to the Euca community.

Positive things that came out of the semester were that we were able to write a lot of useful documentation for the Euca developers and it seems that all of them are very grateful for our assistance. We met some very smart and interesting people who were enthusiastic about us helping out with their project and I really enjoyed meeting them and talking to them over IRC or email. I wish I was able to meet “mull” when he visited because I had a few questions to ask him, but I had several previous engagements that I could not get out of at the last minute. I think that the code we actually did contribute, combined with the documentation we wrote, will be very helpful to new Eucalyptus developers and testers. I hope that the next group to come along and develop for the project is able to use all of our materials and make some large developments to help out the project. I wish all the Euca people the very best and I hope the project is successful for many years to come.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

Today I continued working on the eutester directory. I made some changes to the machine.py file and I’m looking at a few others at the moment. The good news is that we are continuing to make progress on the repository and hopefully it will be finished in another week or so. The bad news is that it’s been a week and only 3 issues are opened so far. I’m hoping everyone is at least looking at these files and trying to make changes, even if they are not very large changes. I should have another push or two by the end of the week, and I know Nate and Marcos have both had a few pushes already, so I’m hoping this will continue and we can finish this up fairly quickly. If everyone on the team does a few changes and commits them, we should be able to get everything done in a week. However, if things continue the way they are, and we average only 4 pushes per week, we may not finish the project entirely. 

I am ambitious about getting it finished, so I think that at the very least, it should be more than halfway done by the end of the week, leaving very little left for our final week of classes. If I finish my directory, and both Nate and Marcos finish theirs, we should be down to about 3 directories I believe, and there are at least 4 more people in the group, so the work should not be difficult to finish. I’ll keep working on my stuff this week, and hopefully I’ll have enough done to move on to another directory, or at least help some other people finish up theirs. I’ll send out an email about finishing this stuff up to my group this week to make sure everyone is on board. Overall, I think we are in decent shape and we should be able to finish everything we set out to do before the end of the semester. 

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

At the end of the last class, we decided to use the built in issue tracker for github to track our progress and also to make sure no two people are working on the same project at the same time. This way, each person can choose which file or which directory they would like to work on and they can open a ticket or issue that corresponds with their project. Once an issue is opened, it is saved on github for the rest of the group to see so they can make a choice regarding what they want to work on. It prevents accidental duplication of work and it also lets the community know exactly what we are working on at that point in time.

As of this point, I have an open issue that corresponds to the eutester/eutester directory and I have added a milestone that lets everyone else know that I have finished the eulogger.py file. Each time I finish a piece, I can add a new milestone to record my progress. I know that Nate also has an issue open, and I believe that Stephen, Coady, and James were all opening issues as well. So by the end of the week, we should know exactly who is working on what part of the project and where they are with it. If not, I may have to open several issues and assign them to members of the group myself via github. However, either way, we will know more as the week goes on and the project should start getting closer to complete!

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

I just submitted my first patch of code! It only contained minor code changes including comments and moving code to a function instead of leaving it out in the open, however it seemed to help significantly. I used a short shell script written by Nate Doe to help me out in linking my local repository to the main github fork for our class. I simply followed the git commands and used the shell script to finalize the commit and push the changes. I asked my group to work on specific classes, so we should have more pushes this week. I also have some more ideas for changes to other classes, so I should have more this week as well. Looking forward to finishing our first directory!

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

This week, I was actually able to not only add some documentation to the eutester code, but I actually made some small code changes. There were a few existing comments in the code that basically said “This code block needs to be moved into a function at some point,” so I went ahead and created the function for it. I worked specifically the eulogger.py class and I added a bunch of comments explaining the code and I was able to actually get the code to compile cleanly after moving the code block into a function. 

I am looking forward to testing the code, but as of now, it is very exciting that I was able to modify and compile the code so far. There is still a lot of work to be done, but I feel confident that we will be able to commit a lot of useful documentation and minor code changes. Even though what we are doing is not very large or innovative, it is still useful to the Eucalyptus community and it feels good to know that we are helping out. I should be ready to commit a patch (or at least push some changes) within the next week. I would like to really finish up some more stuff before committing, however I think I am almost ready to go. I will do my best to have something up by the end of the day, but we’ll see how much gets done. 

The next step will be to finish up the rest of the eulogger directory. I will discuss this with my group and hopefully assign different classes to different people and get several classes committed this week!

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

This week was more looking through code and understanding what is going on than anything else. After we learned about the paramiko and boto modules that we needed to run the Python code, a lot of the code made more sense. We all want to have a pretty good grasp on the code before we actually start doing anything. But this coming week I will actually start to document a little and hopefully I’ll have a patch ready by the end of the week. I would like to start at the top and work my way down through the different files if possible. For example, I would like to start documenting the setup.py in the highest directory before moving into the deeper stuff if possible, but it depends on where the rest of the group is at with the code I suppose. We could start dividing it all up my directory and each person could document an entire folder (or at least a few files in each directory if there are more people). This week I will email my group and find out what they think is best. Either way, we should have some of the documentation done by the end of the week.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

This week we really learned about what goes into the eutester code to make things happen. There were several pieces that we could not figure out right away, but we spent this week doing research and we discovered what each missing piece actually was. For example, we had no idea what “paramiko” meant at first, but we discovered that it is an ssh module that allows Python code to be compiled and run remotely. Paramiko needs to be installed in order to run the eutester package. There were a few other small modules that we did not understand at first, but we eventually figured it all out and posted links on the wiki about everything we came across.

Now, rather than linking a Eucalyptus instance to a kernel like we wanted to do last week, we are able to actually run Eucalyptus images and ssh into them, which will save a lot of time. So, in moving forward, the Eucalyptus team decided that it would be very beneficial if we went through the eutester package and documented all of the code. Most of it is undocumented, so it can be tedious to read through, but we have a fairly strong understanding of it now, so we should be able to document it and contribute our changes to the project. This coming week will involve a lot of code reading and documenting, but it should be very helpful in the end.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

When Neo went down, it was difficult to determine what the next course of action would be since we need the cluster to get Eucalyptus running, and we need Eucalyptus running to actually use eutester. But in the meantime, we worked on several other small projects to get ourselves ready.

First of all, we moved all of our information from our PiratePad to the wiki so that others could start using our material more readily than before. We all made sure we understand the Python code fairly thoroughly as well. On the wiki, we posted info about eutester, getting the repository downloaded, some of the tutorials on Eucalyptus and Python, and even a very comprehensive API for the actual Python code in eutester.

Stephen and I set off to work on figuring out a few key concepts from the code that we are unsure about for this week. There are several code modules in eutester that look like they have to do with specific hardware/operating system instructions, but we want to find out what their jobs are, so we are doing the research this week/last week.

Additionally, we set other members of the team to the task of learning to handle Eucalyptus instances and trying to ssh into them, since we were all originally having problems doing this. Basically, we want people to be able to run their own Eucalyptus image and ssh into it so they can test on their own, or at the very least, just start learning the basics.

This leads into our final (and possibly most ambitious) “assignment” for ourselves this week. We would like to create a Linux kernel that is bundled with a working Eucalyptus image so that we can just upload this kernel whenever we want to test anything and we always know that it will be a working version and it will be compatible with our stuff. Another positive to this would be that since the Eucalyptus servers get wiped every 6 hours, we would already have the image of our personal kernel stored on a local machine and we could simple reupload when the servers get wiped. There is still much to be done before we can understand everything well enough to get to that point, but I think we are on the right track and our group has made very good progress so far in understanding everything they can.

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.

We are about halfway through the semester and it seems like things are moving kind of slowly. I have to admit that I did not get much done this week even though I set some fairly ambitious goals last week. I read up on some more of the Eucalyptus and eutester packages and I really don’t understand all of it. Also, Neo went down this week, leaving Morpheus as the head node in our cluster. This is a pretty decent set back for the class because we need to get euca2ools set up on either Neo or Morpheus (not entirely sure if it matters which one yet) in order to begin running instances of Eucalyptus and testing it using eutester. There is not much we can use eutester for in the meantime, though I and several of my group mates have been brushing up on our Python skills lately to make sure we understand the code thoroughly. This way we will be ready to start testing and writing code as soon as everything is up and running properly. This week I plan to do a little more research and keep an eye out for Neo so that I can get on and install euca2ools (if it is not already set up). Hopefully this week will be a little more productive on my end and a little less harmful for our server…

From the blog trevorhodde by Trevor and used with permission of the author. All other rights reserved by the author.