Working With UTS

Posted on December 2, 2013 by Chad Wade Day, Jr.

This is my first time working with a team that manages networking and other things. At work I was able to plan out the networking with a few others and if we ever needed something done it was usually something we could handle ourselves. You don’t really have the kind of freedom and ease for doing those kind of things when you have to send in requests for a lot of different tasks.

I’m not really complaining, it’s just a different experience. The most important takeaway is to plan out what you need in advance and send out requests as soon as possible. Some things will take more time than others as some tasks need to be verified to ensure network security and they definitely have other tasks to deal with than ours.

One big holdup we had was the networking for the cluster. It took Dillon and I a bit to figure out exactly what kind of networking we could do as well as what we wanted to do. The original cluster had one IP on WSU’s LAN, and then a machine acted as a router and handled a separate LAN just for the cluster. We figured that wasn’t really necessary and it would be easier enough to just have each machine with an IP on WSU’s LAN instead. Initially we decided to just use the next 10 IP addresses from the block we were given for virtual servers. This set-up didn’t work at all, and we figured those IP addresses were on a separate VLAN or something like that. Dillon later attempted to use the IP that was assigned to the server before but that didn’t work. We could acquire IP addresses from DHCP, but we wanted some static IP addresses that we wouldn’t have to worry about IP conflicts due to being in the DHCP pool or anything like that. We then had to ask UTS about getting some IP addresses, and while we waited we would just install the servers using the DHCP addresses. When UTS got back to us they told us that the whole block of the IP address that was given for the cluster before should be available, but we had already tested it and it didn’t work, but we tested it again just in case and still it still didn’t work. So we had to troubleshoot with them, and eventually we found out that the VLAN assigned to that port was wrong and got everything sorted out. It was kind of a hassle overall, but we definitely could have avoided it if we tested IPs and figured them out earlier in the semester.

One thing I was working on individually was Google Apps authentication for the wiki. I’ll go into it more in a post about the wiki, but after setting it up I found that the feature wasn’t enabled for WSU accounts. So we had to see if UTS thought that was a good feature to enable- I’ll give an update once we get that up or not.

We had the IP addresses available to us for the virtual machines in advance, Dr. Wurst had asked about them earlier in the year for the server for the CS401 project so we were all set regarding those.

In one particular case we avoided UTS altogether. When we were moving the new CS server to production, we found that the old server wasn’t assigned the public IP address directly- it was assigned a local IP address. This sounded like a 1:1 NAT set-up, which is essentially where you have local IP addresses for devices on your network and you route public IP addresses to certain IP addresses. So for moving the new CS server to production, we figured just changing the IP address from the old one over and giving the new server the old one’s local IP address we’d be able to put it in production with no issues. We were able to do this and everything worked out well.

Overall I’d say working with UTS was a good thing. It definitely caused a bit of hassle as it wasn’t really what I was used to, but overall a team like that is definitely a good thing to have.

From the blog Five mvs of Doom by Chad Wade Day, Jr. and used with permission of the author. All other rights reserved by the author.

Posted in cs-wsu | Comments Off

First GNU Guile Patch and More Guix Packages

Posted on November 22, 2013 by David Thompson

I have spent some of the last month working on contributing to GNU
Guile and now I can finally say that I have contributed code to the
project. Guile has several hash table implementations: a Guile native
one, SRFI-69, and R6RS. SRFI-69 contains a handy procedure,
alist->hash-table, which allows for a sort of hash literal-like
syntax:

(alist->hash-table
 '((foo . 1)
   (bar . 2)
   (baz . 3)))

However, I prefer to use Guile’s native hash implementation, and
SRFI-69 is incompatible with it, so I decided to port
alist->hash-table. Unfortunately, it was not as simple as writing
just one procedure. The native hash tables actually have 4 different
ways of doing key comparisons: Using equal?, eq?, eqv?,
and user-defined procedures. This is a design flaw because the user
must make sure to use the right procedures at all times rather than
simply set the hash and assoc procedures when creating the
hash table instance.

Below is the final implementation. Since 3 of the 4 variations were
essentially the same, I wrote a short macro to reduce code
duplication.

;;;; Copyright (C) 2013 Free Software Foundation, Inc.
;;;;
;;;; This library is free software; you can redistribute it and/or
;;;; modify it under the terms of the GNU Lesser General Public
;;;; License as published by the Free Software Foundation; either
;;;; version 3 of the License, or (at your option) any later version.
;;;;
;;;; This library is distributed in the hope that it will be useful,
;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;;;; Lesser General Public License for more details.
;;;;
;;;; You should have received a copy of the GNU Lesser General Public
;;;; License along with this library; if not, write to the Free Software
;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
;;;;

(define-syntax-rule (define-alist-converter name hash-set-proc)
  (define (name alist)
    "Convert ALIST into a hash table."
    (let ((table (make-hash-table)))
      (for-each (lambda (pair)
                  (hash-set-proc table (car pair) (cdr pair)))
                (reverse alist))
      table)))

(define-alist-converter alist->hash-table hash-set!)
(define-alist-converter alist->hashq-table hashq-set!)
(define-alist-converter alist->hashv-table hashv-set!)

(define (alist->hashx-table hash assoc alist)
  "Convert ALIST into a hash table with custom HASH and ASSOC
procedures."
  (let ((table (make-hash-table)))
    (for-each (lambda (pair)
                (hashx-set! hash assoc table (car pair) (cdr pair)))
              (reverse alist))
    table))

Not only did I manage to get my code into Guile, I was given commit
access to GNU Guix by the lead developer, Ludovic Courtès. I have
never had commit access to any free software project repositories
besides my own. I feel quite honored, as cheesy as that may sound.

Packages for SDL and SDL2 have been merged into master, and packages
for SDL_gfx, SDL_image, SDL_mixer, SDL_net, and SDL_ttf for SDL 1.2
are pending review.

I like to show off the elegance of Guix package recipes, so here’s
some code:

;;; Copyright © 2013 David Thompson <dthompson2@worcester.edu>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define sdl
  (package
    (name "sdl")
    (version "1.2.15")
    (source (origin
             (method url-fetch)
             (uri
              (string-append "http://libsdl.org/release/SDL-"
                             version ".tar.gz"))
             (sha256
              (base32
               "005d993xcac8236fpvd1iawkz4wqjybkpn8dbwaliqz5jfkidlyn"))))
    (build-system gnu-build-system)
    (arguments '(#:tests? #f)) ; no check target
    (inputs `(("libx11" ,libx11)
              ("libxrandr" ,libxrandr)
              ("mesa" ,mesa)
              ("alsa-lib" ,alsa-lib)
              ("pkg-config" ,pkg-config)
              ("pulseaudio" ,pulseaudio)))
    (synopsis "Cross platform game development library")
    (description "Simple DirectMedia Layer is a cross-platform development
library designed to provide low level access to audio, keyboard, mouse,
joystick, and graphics hardware.")
    (home-page "http://libsdl.org/")
    (license lgpl2.1)))

(define sdl2
  (package (inherit sdl)
    (name "sdl2")
    (version "2.0.0")
    (source (origin
             (method url-fetch)
             (uri
              (string-append "http://libsdl.org/release/SDL2-"
                             version ".tar.gz"))
             (sha256
              (base32
               "0y3in99brki7vc2mb4c0w39v70mf4h341mblhh8nmq4h7lawhskg"))))
    (license bsd-3)))

Much help is needed to package the GNU system for Guix. If you are
interested in helping, please drop by #guix on freenode, we are
friendly. :)

From the blog dthompson by David Thompson and used with permission of the author. All other rights reserved by the author.

Posted in gnu, guile, guix, scheme, WSU | Comments Off

Learning the Hadoop Install

Posted on November 1, 2013 by dillonmurphy10

After discussing with Chad Day the possible solutions for setting up both a Hadoop cluster and an Eucalyptus cluster on the same set of machines, we have concluded we should first attempt to run the services side by side. We are planning on using CentOS 6.4 as the operating system on the machines, and then installing the services we need for the nodes. The nine machines we have now currently have operating systems and packages on them from the 401 class that focused on working with Eucalyptus. Because of this Chad and I will install the latest version, 6.4, of CentOS on the machines and install the necessary packages for the servers.

Before attempting to install the Hadoop software on the CS machines, I am planning to set up a small environment of Hadoop servers on my personal machine using VirtualBox. This gives me a little more flexibility to play with packages and will be easier to access as I am learning the install process and how the head node and data nodes work together. Once I am confident with the install process on the virtual machines also running CentOS 6.4, I will install the packages on the machines that will have both Hadoop and Eucalyptus.

As far as installing the base operating system on the nine machines, Chad and I will install CentOS on the machines this week, hoping to have all the base system on all the machines by the end of this upcoming week. My plan is to have a couple install medias, either several CD’s or USB drives with the CentOS image on it to help the install go a little quicker. After all the machines are set up and I’ve played around in the virtual environment, I will be ready to install the cluster that will be used in the computer science department.

From the blog WSU Comp. Sci. Servers » wsucs by dillonmurphy10 and used with permission of the author. All other rights reserved by the author.

Posted in wsucs | Comments Off

My Year of Open Source – Post Mortem

Posted on October 31, 2013 by Karl R. Wurst

Some of you will remember the post My Year of Open Source from 1 January 2011 – almost 3 years ago – where I made a New Year’s resolution to participate more in FOSS. Here are the goals I listed for myself for that year:

I have four main goals (at this point):

  1. Learn the tools and processes myself by participating in a FOSS project.
  2. Figure out what FOSS tools and processes I can begin to introduce my students to in earlier courses.
  3. Figure out what FOSS experience(s) I can provide my non-CS students.
  4. Find a project (or projects) to place my Senior CS students into in Spring 2012.

Well, it was as successful as most New Year’s resolutions – meaning, not very. Or maybe, not completely. I was (partially) successful at some of those goals, although almost none were completed within the year that I so rashly promised.

Figure out what FOSS tools and processes I can begin to introduce my students to in earlier courses.

This one was somewhat successful, although not until this past June (2013) when I managed to have my summer Introduction to Programming class (all six students!) use git and Bitbucket to collaborate with their lab partners and to submit their work to me for grading. Fresh from that (small-scale) success, I tried to have my Programming for Non-CS Majors class do the same, and ran into some scaling issues. We’re working on the solution for that right now – more in a future post.

My Spring 2013 capstone project course did use git and GitHub for our project developing an app for a Worcester Art Museum exhibit. But my understanding of git was not a good as it could have been and the student use of git was spotty. We also planned to use Pivotal Tracker, but didn’t get very far. We did successfully use IRC, however.

Find a project (or projects) to place my Senior CS students into in Spring 2012.

My Spring 2012 capstone project course worked with Eucalyptus, and had some pretty strong interaction with some of the members of the community, but I think that both the students and I felt we weren’t as successful as we could have been due to some technical issues early on in the course. For Spring 2013, I abandoned working in an existing FOSS project in favor of new development when the Worcester Art Museum opportunity presented itself. We did, however, make our code freely available (https://github.com/CS-Worcester/JILOA)

Figure out what FOSS experience(s) I can provide my non-CS students.

This goal got very little attention, other than my abortive attempt at using git in the Programming for Non-CS Majors course.

Learn the tools and processes myself by participating in a FOSS project.

And I still have not made any real progress in my own participation in a FOSS project.

However, that’s all going to change. Stay tuned for My Year of Open Source v2…

 

From the blog On becoming an Eccentric Professor... » CS@Worcester by Karl R. Wurst and used with permission of the author. All other rights reserved by the author.

Posted in Computer Science Education, CS 401, CS@Worcester, FOSS, My Year of Open Source, Teaching Open Source | Comments Off

IRC: 1, Nemo: 0

Posted on October 30, 2013 by Karl R. Wurst

Our CS 401 Software Development class was canceled on Monday, 11 February 2013 due to ongoing snow removal and cleanup on campus from the Nemo blizzard. (Worcester received 28.5 inches of snow in just about 24 hours.) This is a problem for a class that meets only on Mondays, especially with the next Monday being a holiday.

As soon as the campus closing was announced on Sunday afternoon, I emailed the students to announce that we would replace class the next day with an IRC (Internet Relay Chat) meeting. (Actually, that’s a lie. The first thing I did was panic, then I screamed, then I ranted to my family about the injustice of cancelling my Monday-only class. Then I thought about holding class on IRC…) Here is the message I sent the students on our class listserv:

Campus is closed tomorrow, so we will not have class. We will not have class next week either due to the President’s Day holiday.

This is going to seriously mess up our schedule. I’ll think about how we can carry on in the two weeks.

Let’s try to hold an IRC chat tomorrow during class time (2:00pm-4:30pm). I’ll send out instructions on installing (optional) and using an IRC client later tonight. I have instructions already written up, I just have to find them, possibly update them, and send them out.

Holding class on IRC would be a little bit of a challenge since the students had not used IRC yet, so this would have to serve as both an IRC familiarization exercise and a useful meeting. I sent them the following message to prepare them:

We are going to meet today on IRC (Internet Relay Chat) at 2:00pm.

You should read through this in advance so that you are prepared. Especially if you are going to install an IRC client – you will need time to set it up. I suggest trying this out at least 1/2 hour in advance to make sure you get the connection working. I’ll stay on IRC all day so you can try out chatting.

You have two choices for connecting to the IRC server:

  1. Install an IRC client. There are many available, you may want to try a few to see which you like the best. Some are standalone applications, and some are browser plugins (like Chatzilla for Firefox.) I’ve heard that mIRC is the most popular for Windows, I use Colloquy on the Mac.
    Here are some of the most important settings you will need. How you set these will depend on your client. You will want to install your client and do the setup in advance of our meeting, so you aren’t late.

    1. Server: irc.freenode.net
    2. If you can set a port, you may want to use 7000 since it can be used for an SSL connection.
    3. Nickname: Choose your own*
    4. Channel: ##WSU-CS401
  2. Use the webchat page on freenode: https://webchat.freenode.net
    1. Nickname: Choose your own*
    2. Channels: ##WSU-CS401
    3. Complete the reCAPTCHA
    4. Connect

* You may want to register your nickname, so that no one else can use it. That way we can all get used to looking for a specific nickname for you. See the instructions: http://freenode.net/faq.shtml#registering

IRC Resources

The most important commands which chatting:

  • /SERVER new-server-hostname
  • /NICK new-nickname
  • /QUIT
  • /JOIN #channelname
  • /ME does something
    This command is used for saying that you are doing something like:
    /ME is looking for that information in my email
  • /LEAVE

Chatting:

  • If you want to address your comments to everybody, just type your comment and hit return.
  • If you want to address your comments to a specific person, type their nickname followed by a colon, then your message. E.g.

         kwurst: I have the answer to your question

I had created a course-specific channel on freenode last spring, so we could use that channel, but to hold a useful meeting, felt that it would be vital to have a MeetBot running to take minutes. I could have used used the #teachingopensource channel, which has zodbot installed, but then the minutes would be saved on Fedora’s website, rather than ours. So I decided to install Supybot with the MeetBot plugin on our own server here.

I managed to get MeetBot installed (mostly – gives me an error message for every meeting command I give, but then does it anyway) and we had a very successful meeting for a class of IRC newbies: http://cs.worcester.edu/kwurst/wsu-cs401/2013/wsu-cs401.2013-02-11-21.13.html

From the blog On becoming an Eccentric Professor... » CS@Worcester by Karl R. Wurst and used with permission of the author. All other rights reserved by the author.

Posted in Computer Science Education, CS 401, CS@Worcester, FOSS, Teaching Open Source | Comments Off

WSU Comp. Sci. Servers » wsucs 2013-10-25 11:37:42

Posted on October 25, 2013 by dillonmurphy10

I am pleased to report a seamless move to the new wiki and blog server for the Worcester State University Computer Science Department! After collaborating with Dr. Karl R. Wurst and Chad Day, we moved the ip of the old wiki and blog server to the new server and assigned another ip to the old one. After running tests on both the wiki and the blog, cs.worcester.edu/wiki and cs.worcester.edu/blog are working as expected.

At this point in the semester, the blog and wiki have been migrated to an updated operating system. However, we are still looking into some possibilities on how to secure the wiki from potential spammers. One such way is to look into the code and limit email registration to the worcester.edu domain, which will be looked into by Chad. The git server is up and running and the CS department is in the process of testing it for usage in classes for homework submission and a good way to teach version control early in the major.

Looking forward, Chad and I will be building two clusters with the Computer Science machines, an Eucalyptus cluster and a Hadoop cluster to be used throughout the semester. Before we install the software on the machines, we must investigate the best solution for sharing hardware. We currently have nine machines at our disposal, and we must partition them for the clusters. The possibilities have investigated so far include:

  1. Splitting the physical allotment in half –  4 machines for each cluster
  2. Installing the correct services along side each other, each cluster share the nine machines and share resources
  3. Installing a virtual host on the machines and running two machines on each host, each cluster would have 9 machines and share resources, but would remain independent of each other.

We should have a strategy and start formatting the host machines by the end of next week, followed by another update.

From the blog WSU Comp. Sci. Servers » wsucs by dillonmurphy10 and used with permission of the author. All other rights reserved by the author.

Posted in wsucs | Comments Off

Migration of cs.worcester.edu/blog

Posted on October 24, 2013 by dillonmurphy10

One of the tasks for this fall semester for the server migration/creation study was to move the Worcester State Computer Science blog to a new server alongside a new wiki platform. For both servers, WordPress will be used to manage the blogs, and all posts will be aggregated from other sites. In moving the blog to a new server the following steps were followed:

  1. Investigate the structure of the original server
    1. Web directory stored in a separate lampp folder
    2. MySQL server run from lampp folder
  2. Choose a structure for the new server
    1. Use a standard install of MySQL and Apache
    2. Set up Apache configuration that will work when the ip is changed
    3. Create a MySQL database that matches old server
    4. Create a MySQL user with the same credentials and permissions as the old server
  3. Move the old web directory to the new server
  4. Move the database to the new server
    1. Make a database dump of the WordPress database
    2. Move dump to new server and restore to newly created database
  5. Check that web directory works as expected with migrated database settings
  6. Find admin and home URLs settings in database and update them through the database
    1. This allows you to use the admin page on the new server instead of redirecting to the old admin page
    2. Change any necessary settings on the new page
  7. Run a final test to make sure everything is up to date

Following the steps above, the migration of the server was straightforward and was about ready for an ip change. The only issue with the migration was the updating of the pages (ex. Order of the Rubber Duck) did not carry over. After investigating the database for possible URL conflicts a few settings were changed with URL and as soon as a scheduled check was run by WordPress, the pages were restored. 

A final check was made with the blogs, navigation and aggregation are all set, and a scheduled update of the ip address to match cs.worcester.edu will occur tomorrow, October 25, 2013. After the ip has been migrated a thorough check will be made and if everything is a success, the new server will be used for the blogs.

From the blog WSU Comp. Sci. Servers » wsucs by dillonmurphy10 and used with permission of the author. All other rights reserved by the author.

Posted in wsucs | Comments Off