CS@Worcester

Although this is an independent study, at times I worked with Dillon Murphy. For the most part we worked on separate tasks, but there were a few we worked together on.

We did work together on all of the virtual server installs, also working with Dr. Wurst. Dillon and I were pretty familiar with server installs, but since the new CS server was being used for the Wiki (which I was focusing on), and the blogs (which Dillon focused on), it made sense to plan out and figure out what we needed. As I said in a previous post, MediaWiki relied on an Apache/PHP/MySQL setup, and luckily for Dillon WordPress relied on that as well.

During our individual projects, we would shoot ideas off of each other if we ran into issues. For example, for moving over WordPress, Dillon wasn’t too sure how to approach it but since I was familiar with PHP/MySQL applications I suggested that it was likely all he really needed to do was dump the database and restore it on the new server and then move over the code. There were a few issues with configuration files pointing to old URLs, but other than that everything worked out.

We worked together on getting the new CS server up, which worked out thanks to some guesswork and brainstorming to figure out the best way to do it.

Getting the cluster with fresh installs of CentOS was another collaborative effort seeing as we planned to use them for our next projects, Dillon focusing on Hadoop and myself focusing on Eucalyptus. We had a few ideas on how to utilize the cluster for both platforms. One suggestion involved installing a virtual machine hypervisor on each machine and having two Virtual Machines on each, one for Hadoop and one for Eucalyptus. Another suggestion was splitting the machines between each project. The suggestion we went with, after Dillon said it looked like it would work out, was having one CentOS install on each machine and installing both on each machine.

Once we figured out what we wanted to do with the network and resolved the issues with it, Dillon focused on getting the servers up. We finally got that up and running, so next is focusing on the final solo projects.

While Dillon focused on that and a bit before, I focused on getting GitLab working to usable conditions. Dillon and Dr. Wurst helped test and ensure that everything was working as it should.

It’s really helpful to work in a team, even if each member is focusing on a separate individual project. Being able to bounce ideas off of each other and resolve things more quickly. Working on your own can be productive for its own reasons, but it never hurts to have a helping hand available.

From the blog Five mvs of Doom by Chad Wade Day, Jr. and used with permission of the author. All other rights reserved by the author.

I missed a few things I wanted to go over with GitLab, and I have an update on trying out CACerts.

I didn’t really go over GitLab or git itself and why it’s important.

To understand GitLab, first you have to understand git. Git is a revision control software for plain text files. What this means is, you edit the code, commit your changes, and edit it again, and your code changes are tracked. Linus Torvalds, the creator of the Linux kernel created it when he wasn’t happy with the other software that was available. Git is one of my personal favourite tools to use, and has helped me with development a ton. I’ve definitely made poor choices in improving code or deleting code I didn’t like and want to undo that- with proper use of git this is trivial. One really advantageous feature of git is that it’s decentralized- when you clone a project with git, all its contents are self contained on your computer and if you want to update with a remote server you can pull the changes from the server or push your changes to the server. You can even have multiple remote repositories to push and pull from, though in practice you won’t use that feature too much. Another useful feature is you can set the remote repository to be another location on your local machine- this can be useful if you’re away from the internet and worried about accidentally ruining your working directory.

GitLab is probably the best free software alternative to Github. Github is a web front-end for remote git repositories, bring a lot of additional features to git such as editing your code from the web, forking other projects and pushing changes back to main projects (useful for feature development), easy account and group management, and of course the advantage of keeping your code stored on a remote server in case of issues with your local machine. For an example of a Github project, you can check out the CS-401 project from Spring 2013 that I was a member of. GitLab offers pretty much all of Github’s main features and a familiar interface. The big advantage of GitLab over Github is that we can store the data on WSU’s location, as well as private repositories, which are a paid feature of Github.

So as far as our GitLab install goes, last night I looked into using a certificate from CACert. Turns out, to apply for a CACert you need to own the domain, so that was quickly scrapped. I don’t think we can get HTTPS working at this point. I tried a few things with the configuration but it *seems* like the issue is the self-signed certificate and obtaining a legitimate certificate I don’t think is possible. This isn’t a huge issue, though with SSH you have to install keys on the server so it requires a bit more documentation.

What needs to be done with the server from here? I think all we need to do is resize the partition and virtual hard drive the server is on. I believe initially we set it to a 16gb install, but if students are going to use it for many assignments in their classes I feel like that’d fill up pretty quickly.

From the blog Five mvs of Doom by Chad Wade Day, Jr. and used with permission of the author. All other rights reserved by the author.

Where do I start with GitLab?

We had to install 3 servers to finally get this up and running and I’m still not completely satisfied with it. It’s frustrating to set up. It’s great software when you get it up and running, there’s no doubt about that, but it sure is a pain to get up and running.

Before I had done any work on setting up a GitLab server for WSU’s CS department, I had a bit of experience with it. I had tried to set up GitLab on some distro of Linux before for playing around with at home following the install guide but didn’t get anything working with it. We had a GitLab server running at my work at one point using the BitNami installer, but there were a few issues with it so we scraped it for something else.

The first attempt was a GitLab server install with the BitNami installer. We set up a CentOS and I ran the BitNami installer and got something up and running pretty easily. However, there was the most bizarre issue I had seen- for some strange reason pages would just show up as blank in Firefox unless you refreshed. I couldn’t find anything covering the same issue at all. I didn’t think something with an issue like that was worth keeping around, so we scrapped that and set up another CentOS server.

Here I decided to find a guide for installing GitLab on CentOS instead of relying on an installer or anything like that. In hindsight this probably wasn’t the best idea, the official install guide recommends Ubuntu and we probably would have had a much easier time just going with Ubuntu server. However, we had CentOS on hand, and I much prefer using CentOS for a server as it has some pretty good default configuration files for a lot of applications.

I didn’t have any issues with setting GitLab up initially using this guide, but afterwards a few problems became apparent. First off, neither SSH or HTTPS cloning worked. After a bit of hassle we got SSH working and were fine with that and decided that things were good to go. We sent in the request to UTS and got a public IP and subdomain for the server. However, we quickly discovered that sending emails didn’t work. It turned out that it was trying to send emails from gitlab@<localip> which get rejected. Unfortunately, I couldn’t find anywhere to change where it was sending emails from. I changed configuration files all over the place but had no success at all in fixing that. It got to the point where I just settled on doing a complete reinstall with the subdomain in mind which would hopefully fix all the issues. I eventually decided that uninstalling GitLab would be too much of a hassle on its own, so I would have to make a completely new server install. After setting my mind on doing that I spent a bit more time trying to get the current one installed and totally broke it, and didn’t care enough to fix it.

So finally, the last install, the one we have up and running. Using the same guide as before but documenting any oddities I found so we’d have a reasonable installation guide, I got it running and working to a pretty reasonable state, but with a few bumps along the way. First off, the guide recommends running a test to make sure everything’s running before you set up the webserver. It gives an error if the webserver isn’t up and running, but I had no indication this was caused by the webserver not being set-up. I decided to just install the webserver and hope it fixed it, and thankfully it did. Secondly, I tried Apache as the webserver, as the guide had the option of using nginx and apache. nginx seems like what GitLab was developed for so I used that for the previous installs, but for this one I decided to go with what I was familiar with. Didn’t get anywhere with it, and just went back to nginx and go that working pretty easily. Thirdly, the SSH issue we had before came back. This issue had two parts: First off the SELinux permissions aren’t set correctly for the ssh keys file. I’m not sure the exact cause for this, but I think it’s because GitLab creates the file itself if one doesn’t exist, so the correct permissions don’t exist for it. It’s an easy fix if you can figure out that’s the issue, but there’s really no clear indication that’s the issue so I was stumped for a while until I decided to stop looking for GitLab related issues with SSH and instead CentOS related issues with SSH. It was a really simple command, restorecon -R -v /home/git/.ssh We didn’t have this issue with the other server installs, but I think in those cases I disabled SELinux altogether and this time I was trying to get it to work with it. The other issue was that CentOS by default locks accounts without passwords. The GitLab docs recommend editing the /etc/shadow file manually, but I found a command that accomplishes the same thing without editing important configurations: usermod -U git. Email was working fine too.

I still need to get HTTPS cloning working. I believe it’s an issue with using a self-signed certificate, so I’m going to try out using a CACert certificate and seeing if I can get it working from there.

From the blog Five mvs of Doom by Chad Wade Day, Jr. and used with permission of the author. All other rights reserved by the author.

In my first post of the semester, we set up a CentOS server that we planned to have be the new CS server. The old one was running a dated version of Redhat, and with the success of the server that was set up for CS401, the latest CentOS seemed like the way to go.

The tasks were split up between myself and Dillon Murphy. Dillon worked on moving over the blogs and I focused on getting a new wiki up.

The old wiki had been plagued by spambots and used a different software so Dr. Wurst figured just setting up MediaWiki and moving over what he could would be a good direction to go in.

MediaWiki is a PHP/MySQL wiki software built and designed for Wikipedia. Although it was built for Wikipedia, it’s a free software application and as such can be freely used by anyone. I’m not sure if it’s the most popular wiki software, but it’s definitely the one I’ve seen used around the most. Aside from Wikipedia, one notable site that uses the MediaWiki software is Wikia where users can easily create MediaWiki installs for arbitrary uses. It’s also the wiki software of choice we use at my work, which we mainly use for documenting various important information such as networking info, what’s running on servers, troubleshooting tips, and that sort of content.

A PHP/MySQL application relies on PHP for the code and MySQL for the database. These applications are usually pretty trivial to install, so long as you have php, mysql, and a webserver running. I’ve only ever really worked with Apache as a webserver, so we have the common Apache/PHP/MySQL set-up for this server. This is also the set-up one would use for WordPress, so it worked well for our multi-purpose server.

I’m pretty familiar with MediaWiki in a general sense, I remember installing it and playing around with it years ago because I found wikis interesting. I remember years ago it wasn’t as easy to install as it is now, with its main focus being to make Wikipedia better, but nowadays its really trivial to set up and get running. (It could have been that I wasn’t as used to installing web applications as I am now though, haha)

So there shouldn’t be any issues setting this up right?

Luckily there weren’t any issues getting this up and running. I had also recently set-up Wikipedia styled URLs for pages at work, so I was able to get that working with no problem either. After setting everything up, I created a user for Dr. Wurst and a user for Dillon to test that everything was working. Everything seemed to be running with no issues, so mission success!

The next step was to figure out how we were going to handle user authentication. Ideally, we don’t want it possible for spam bots to sign up at all due to what happened to the older wiki. Initially I set it up so users can not register at all until we figured things out. We decided against using LDAP authentication since we didn’t want to bother UTS too much. Originally we were thinking of editing the code to only accept registrations from the @worcester.edu domain- this probably wouldn’t be too difficult to implement, but I found that the Google Apps Authentication plugin accomplished pretty much the same thing, restricting logins to Google Apps users from a specific domain using OpenID authentication. Users would be created automatically on login if they didn’t exist, so it was an ideal solution. Unfortunately I found that this authentication method wasn’t enabled for the worcester.edu domain, so we sent in a request to UTS to see if they could enable it.

While we waited for an update on if it was a good idea to enabled and if its enabled, Dr. Wurst asked about setting up subpages. Subpages is a feature not enabled by default, but allows you to make pages with the title PAGE/Subpage which are pages by the name of “Subpage” that link to the original page “PAGE” right at the top for easy navigation. This was pretty trivial to setup, and it’s a pretty nice feature to have.

We also moved the server from testing to production as Dillon had also finished moving over the blogs. All we really needed to do for this was take the old server of its current local IP and move the new server to that IP and the domain and public IP would point to it. We weren’t 100% sure it would work out this way, but it seemed like that was the way it would work to me and we could always undo the changes if there were problems and send in a request to UTS to figure things out, but luckily everything worked out and the server was live.

Earlier today I got word that the OpenID authentication feature was enabled for the worcester.edu domain, so I enabled the plugin again and everything worked out as it should. This lead to two issues though: First of all it wouldn’t let me edit permissions of users with an @ in the name as it treats editing those users as editing users from another wiki for some reason. I found a pretty easy fix for that by changing what delimiter is used for that, from @ to #, allowing editing the @worcester.edu users as normal. The second issue was that the old accounts couldn’t be used any more, so I just redirected their user pages to the new accounts. Not the cleanest fix, but I looked into changing the users who made each contribution over and it seemed like a lot of work for an easy task, but it made sense for MediaWiki to do that as having it possible for people to “steal” contributions on a big site like Wikipedia wouldn’t be good for anyone.

As far as the Wiki goes, I’m pretty satisfied in how it turned out.

From the blog Five mvs of Doom by Chad Wade Day, Jr. and used with permission of the author. All other rights reserved by the author.